In‑Depth Overview of NVIDIA Grace Hopper Superchip Architecture

The NVIDIA Grace Hopper Superchip architecture is the first truly heterogeneous acceleration platform designed for high‑performance computing (HPC) and AI workloads, combining the strengths of NVIDIA Hopper GPUs and Grace CPUs into a single superchip with a unified programming model.

Key Architectural Features

Grace CPU: Up to 72 Arm Neoverse V2 cores (Armv9.0‑A ISA) with 4×128‑bit SIMD units, 117 MB L3 cache, up to 512 GB LPDDR5X memory delivering 546 GB/s bandwidth, 64 PCIe Gen5 lanes, and a scalable consistency fabric (SCF) providing up to 3.2 TB/s cache bandwidth.

Hopper GPU: Up to 144 streaming multiprocessors featuring fourth‑generation Tensor Cores, Transformer Engine, DPX, and 3× FP32/FP64 performance compared to A100, with up to 96 GB HBM3 memory (3000 GB/s) and 60 MB L2 cache.

NVLink‑C2C Interconnect: A coherent, high‑bandwidth (up to 900 GB/s total, 450 GB/s per direction) chip‑to‑chip link that provides memory consistency between CPU and GPU, enabling oversubscription of GPU memory and direct access to the full CPU memory pool.

NVLink Switch System: Supports up to 256 Grace Hopper superchips, delivering up to 115.2 TB/s full‑mesh bandwidth, far surpassing traditional PCIe‑based systems.

Programming Model and Productivity

The platform offers a simple, unified programming model that abstracts away explicit memory management. Developers can use familiar languages such as ISO C++, ISO Fortran, and Python, as well as models like OpenACC, OpenMP, CUDA C++, and CUDA Fortran, all compiled by the NVIDIA CUDA LLVM compiler API.

Hardware‑assisted memory consistency and native atomic operations enable lightweight synchronization primitives across CPU and GPU threads, reducing the need for manual data movement and improving performance.

Extended GPU Memory (EGM)

Through NVLink‑C2C and the NVSwitch fabric, the superchip can access up to 150 TB of system memory, allowing GPU threads to address large datasets that exceed on‑chip HBM capacity.

Performance Highlights

Grace CPU delivers up to 2.5× higher performance and 4× lower energy consumption compared to AMD Milan 7763.

Hopper GPU achieves up to 9× faster AI training and 30× faster inference on large language models compared to the previous A100 generation.

NVLink‑C2C provides 7× higher bandwidth than x16 PCIe Gen5, with significantly lower energy per bit transferred.

Overall, the NVIDIA Grace Hopper Superchip combines massive compute power, high‑bandwidth memory, and a developer‑friendly programming environment to accelerate the most demanding AI and HPC applications.