Understanding High‑Performance Computing (HPC): Principles, Architecture, and Terminology

High‑performance computing (HPC) uses supercomputers and parallel processing techniques to complete time‑consuming tasks quickly or to run many tasks simultaneously.

In HPC, information is processed mainly in two ways: serial processing performed by the central processing unit (CPU), where each core handles one task at a time, and parallel processing that can employ multiple CPUs or graphics processing units (GPUs) to execute many operations concurrently.

GPUs, originally designed for graphics, excel at performing arithmetic on data matrices and are therefore well‑suited for parallel workloads such as machine‑learning tasks like object detection in videos.

Breaking the limits of supercomputing requires diverse system architectures; most HPC systems interconnect multiple processors and memory modules with ultra‑high‑bandwidth links to enable parallelism, and many combine CPUs and GPUs in heterogeneous computing configurations.

The performance of computers is measured in FLOPS (floating‑point operations per second). By early 2019, top‑tier supercomputers could achieve 143.5 × 10¹⁵ FLOPS, classified as “peta‑scale.” In contrast, high‑end gaming desktops reach about 1 × 10⁹ FLOPS, making them over a million times slower. The next milestone, “exa‑scale” (10¹⁸ FLOPS), will be roughly a thousand times faster than peta‑scale.

Achieving exa‑scale performance demands massive data throughput; system design must consider memory bandwidth and interconnect latency because data must be continuously fed to processors.

Key terminology includes:

High‑performance computing (HPC): a broad class of powerful computing systems ranging from a single CPU with several GPUs to world‑leading supercomputers.

Supercomputer: the most advanced HPC system, defined by ever‑increasing performance benchmarks.

Heterogeneous computing: architectures that combine serial (CPU) and parallel (GPU) processing capabilities.

Memory: storage within an HPC system designed for rapid data access.

Interconnect: the system layer that enables communication between processing nodes, existing at multiple levels in supercomputers.

Peta‑scale: systems designed to perform 10¹⁵ FLOPS.

Exa‑scale: systems designed to perform 10¹⁸ FLOPS.

Why pursue HPC? From a system perspective, it integrates resources to meet growing performance and functionality demands; from an application perspective, it decomposes workloads to enable larger‑scale or finer‑grained computation; and it solves scientific and engineering problems that require intensive numerical simulation and modeling.