From Simple Loops to SIMD: The Evolution of Parallel Computation in CPU Design

In a storytelling style, the author introduces "A Q", a worker in CPU core 1 of an 8‑core, 16‑thread processor, who together with other functional units (instruction fetch, decode, and write‑back) executes a simple loop that adds 1 to each element of an integer array.

The loop, implemented as void array_add(int data[], int len) { for (int i = 0; i < len; i++) { data[i] += 1; } } , is painfully slow because each iteration fetches a single element, performs the addition, and writes it back.

During a post‑work meeting, the team discusses how to improve performance by processing multiple elements per iteration, raising questions about variable increments, different arithmetic operations, and the need for a new instruction set.

The discussion evolves into a broader consideration of parallel computing, prompting the involvement of representatives from all eight CPU cores. The leader suggests that the problem is essentially one of parallel execution.

The team proposes borrowing larger registers from the floating‑point unit (FPU) and creating a new integer‑oriented SIMD extension called MMX, with registers MM0‑MM7 capable of holding multiple integers simultaneously.

They define SIMD (Single Instruction Multiple Data) and explain how the new MMX instructions dramatically speed up integer array operations.

Two practical issues arise: the need to switch between SIMD and FPU modes because they share registers, and the limitation of MMX to integer operations while modern workloads increasingly require floating‑point parallelism.

To address these, the team expands the architecture with the SSE instruction set, adding eight 128‑bit XMM registers that no longer share resources with the FPU, and later introduces AVX with 256‑bit registers, enabling high‑performance parallel processing of both integer and floating‑point data.

Throughout the narrative, images illustrate the CPU workers and the evolving instruction sets, reinforcing the conceptual transition from a simple serial loop to sophisticated SIMD techniques.