CPU Performance Optimization Using Top‑Down Micro‑architecture Analysis (TMAM)

During development we often care about service performance, but performance tuning is difficult and time‑consuming. Using proper methodologies and tools can quickly locate bottlenecks and guide targeted optimizations.

The hardest part is identifying the critical bottleneck. For native C++ programs developers usually rely on tools such as perf or bcc . This article focuses on CPU‑centric tuning – extracting maximum throughput from the processor.

Example program (a simple accumulation loop) is used to illustrate the analysis:

#include
#define CACHE_LINE __attribute__((aligned(64)))
struct S1 {
  int r1;
  int r2;
  int r3;
  S1(): r1(1), r2(2), r3(3) {}
} CACHE_LINE;

void add(const S1 smember[], int members, long &total) {
  int idx = members;
  do {
    total += smember[idx].r1;
    total += smember[idx].r2;
    total += smember[idx].r3;
  } while(--idx);
}

int main(int argc, char *argv[]) {
  const int SIZE = 204800;
  S1 *smember = (S1 *)malloc(sizeof(S1) * SIZE);
  long total = 0L;
  int loop = 10000;
  while(--loop) {
    add(smember, SIZE, total);
  }
  return 0;
}

Compile and run with:

g++ cache_line.cpp -o cache_line ; task_set -c 1 ./cache_line

The program reaches 99.7% CPU utilization on a single core, but the question remains: is all the work useful? Where is the remaining optimization potential?

CPU pipeline overview

Modern CPUs follow the classic fetch‑decode‑execute‑write‑back stages. Intel CPUs implement multi‑stage pipelines (typically 5‑stage, up to 15‑stage in newer designs) that allow several micro‑operations (uOps) to be in flight simultaneously.

TMAM (Top‑down Micro‑architecture Analysis Methodology) classifies pipeline slots into four top‑level categories:

Retiring – uOps that complete successfully and retire.

Bad Speculation – uOps that are discarded due to mis‑predicted branches or other speculation failures.

Front‑End Bound – slots stalled because the front‑end cannot supply enough uOps (fetch, decode, dispatch).

Back‑End Bound – slots stalled because the back‑end lacks required resources (cache, execution units, memory).

Only Retiring should be high; the other three should be as low as possible. Intel provides reference ratios for typical workloads.

How to map observed stalls to TMAM categories

Performance tools such as Intel VTune or the open‑source pm-tools report the percentage of slots spent in each category. A decision‑tree (shown in the original article) helps assign each uOp to the appropriate bucket.

Optimization guidance per category

Front‑End Bound

Reduce code footprint using compiler optimizations (e.g., -O2/-O3 , -fomit-frame-pointer ).

Leverage macro‑fusion by using unsigned loop counters.

Apply profile‑guided optimization ( -fprofile-generate , -fprofile-use ) and place hot code together ( __attribute__((hot, code)) ).

Unroll small loops ( -funroll-loops ) and simplify conditional logic to improve branch prediction.

Back‑End Bound

Optimize data structures for cache‑line alignment.

Avoid false sharing in multi‑threaded code.

Reduce algorithmic memory traffic and increase instruction‑level parallelism.

Example of proper cache‑line alignment:

#define CACHE_LINE __attribute__((aligned(64)))
struct S1 {
  int r1;
  int r2;
  int r3;
  S1(): r1(1), r2(2), r3(3) {}
} CACHE_LINE;

Benchmarking shows a significant drop in cache‑line related stalls when the structure is aligned.

Bad Speculation

Use GCC built‑ins for likely/unlikely branches:

#define likely(x) __builtin_expect(!!(x), 1)
#define unlikely(x) __builtin_expect(!!(x), 0)
if (likely(condition)) {
  // hot path
}
if (unlikely(condition)) {
  // cold path
}

Avoid indirect jumps, virtual calls, and large switch statements that increase BTB pressure.

Result

After applying the above recommendations, the example program’s performance improved from 15 s to 9.8 s (≈34 % faster). Retiring increased from 66.9 % to 78.2 %, while Back‑End Bound dropped from 31.4 % to 21.1 %.

These figures demonstrate how TMAM‑driven analysis can systematically uncover and eliminate CPU bottlenecks.