Choosing the Right Number of Threads for Java Applications: CPU Utilization and Performance

Many online posts suggest simple formulas for setting thread numbers—CPU‑bound programs: cores + 1 , I/O‑bound programs: cores × 2 —but the author questions their reliability and conducts hands‑on experiments.

First, a busy‑loop Java program demonstrates that a single thread can drive a CPU core to 100 % utilization:

public class CPUUtilizationTest {
    public static void main(String[] args) {
        // dead loop, does nothing
        while (true) {
        }
    }
}

Running this on a 12‑core CPU shows one core fully utilized. Adding more threads (2, 4, 6, …) sequentially fills additional cores until the number of threads equals the core count; beyond that, all cores stay at 100 % but the overall load (e.g., 22 vs 11) rises, indicating more context‑switching.

The author then introduces I/O‑wait periods by inserting Thread.sleep(50) after every 100 million empty loops, simulating I/O latency. With a single thread, the active core’s utilization drops to about 50 %. Increasing the thread count to 12, 18, and 24 shows that while each core’s utilization approaches 100 % again, the total system load grows, confirming that I/O‑bound threads allow higher concurrency without saturating the CPU.

Key observations from the experiments:

A CPU‑bound thread can fully occupy a core; the maximum useful number of such threads equals the core count.

Running more CPU‑bound threads than cores leads to excessive context switches, higher load, and diminishing returns.

I/O‑wait periods free the CPU, allowing the scheduler to run additional threads and improve overall utilization.

The higher the I/O‑wait frequency or duration, the lower the per‑core utilization, but more threads can be kept busy.

Referencing "Java Concurrency in Practice," the article presents a formula for estimating required threads to reach a target CPU utilization:

threads = cores × targetUtilization × (1 + waitTime / computeTime)

Applying the formula to a 12‑core machine with a 90 % target and a 50 ms wait per 50 million operations yields roughly 22 threads. Testing this configuration shows an actual utilization of about 80 % due to extra context‑switch overhead.

In real‑world Java services (e.g., Spring Boot with Tomcat), many other threads already exist (web server, GC, background tasks). Therefore, a pure formula rarely gives an accurate thread‑pool size; systematic testing against concrete goals—desired CPU usage, acceptable GC pause frequency, throughput requirements—is essential.

The recommended practical approach is:

Identify external factors (other processes, JVM internal threads).

Set clear performance targets (CPU utilization, GC behavior, latency).

Iteratively adjust the thread‑pool size and measure the impact.

Ultimately, the author suggests using the number of CPU cores as a sensible default for most simple asynchronous scenarios, while emphasizing that the optimal thread count must be derived from the specific workload and empirical testing.