Understanding AI Chip Architecture: How ASIC Accelerators Differ from CPUs and GPUs

Recently, a two‑year‑old Chinese AI‑FPGA startup was acquired by Xilinx, highlighting the rapid growth of AI chip companies. Most readers are unfamiliar with AI chip architectures, prompting this overview.

AI chips are typically ASICs designed specifically for AI algorithms such as CNNs and RNNs, which mainly perform massive matrix multiplications and additions. General‑purpose CPUs and GPUs can run these algorithms but are slower and less power‑efficient, making them unsuitable for real‑time applications like autonomous driving or mobile facial recognition.

The article compares computational capabilities: a high‑end CPU (e.g., IBM POWER8) can achieve roughly 64 Gops, while Google's TPU1, with a 256×256 systolic array of 64 k MAC units running at ~700 MHz, can reach about 90 Tops—several orders of magnitude higher. Real‑world utilization is lower due to memory bandwidth limits and control overhead.

Other AI accelerators such as Cambricon's DianNao and various FPGA‑based solutions are also mentioned, illustrating the trend toward specialized hardware for deep‑learning workloads.

Historical notes trace AI computation from early CPU‑based neural networks, through the GPU boom, to modern ASICs and NPUs, emphasizing that each platform serves different use cases.

Finally, the article includes a disclaimer and promotional material for an e‑book collection on architecture topics.