Unlock Massive Concurrency in Java with Virtual Threads and Structured Concurrency

1. Virtual Thread Overview

Virtual threads are lightweight threads introduced as a preview in JDK 19 and finalized in JDK 21. They are cheap compared to platform threads, can be created in large numbers, and are not pooled; each task typically gets its own virtual thread. Their lifetimes are short and call stacks shallow, making them ideal for single HTTP calls or JDBC queries.

Unlike platform threads, virtual threads are instances of java.lang.Thread that run Java code on underlying OS threads without binding to a specific OS thread, allowing many virtual threads to share a few OS threads.

Virtual threads follow the familiar per‑request thread model while improving hardware utilization, requiring no new concepts but encouraging developers to abandon the habit of limiting thread counts due to high cost.

2. Traditional Request Thread Model

In the classic model, each concurrent request is handled by a dedicated thread for its entire lifetime. To increase throughput, the number of threads must grow proportionally, but OS threads are expensive, limiting scalability. Thread pooling mitigates creation cost but does not increase the total number of threads, so throughput remains far below hardware limits.

3. Using Virtual Threads

Method 1: Executor per task

<code>// Create an executor that starts a new virtual thread for each task
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    IntStream.range(0, 10_000).forEach(i -> {
        executor.submit(() -> {
            Thread.sleep(Duration.ofSeconds(1));
            return i;
        });
    });
}
</code>

This example runs 10,000 tasks that each sleep for one second; the JDK may use only a few OS threads.

Method 2: Direct virtual thread creation

<code>// Create and start a virtual thread
Thread.ofVirtual().name("pack").start(() -> {
    System.out.printf("%s - task completed%n", Thread.currentThread().getName());
});

// Unstarted virtual thread
Thread virtual = Thread.ofVirtual().name("pack").unstarted(() -> {
    System.out.printf("%s - task completed%n", Thread.currentThread().getName());
});
virtual.start();
</code>

Method 3: ThreadFactory

<code>ThreadFactory tf = Thread.ofVirtual().factory();
tf.newThread(() -> {
    System.out.printf("%s - task completed%n", Thread.currentThread().getName());
}).start();
</code>

Method 4: Static helper

<code>Thread.startVirtualThread(() -> {
    System.out.printf("%s - task completed%n", Thread.currentThread().getName());
});
</code>

4. Virtual Threads vs Traditional Thread Pools

Code examples show that three virtual threads start almost simultaneously and finish in about one second, whereas using a fixed pool of one platform thread processes tasks sequentially, taking three seconds.

5. Use Case – Remote Service Calls

A Spring Boot controller defines three endpoints (/userinfo, /stock, /order) that each sleep for a few seconds. A service method uses a virtual‑thread‑per‑task executor to invoke the three endpoints concurrently, achieving near‑linear speedup compared with a traditional fixed‑size pool.

6. Structured Concurrency (Preview)

Structured concurrency groups related tasks into a single unit, simplifying error handling, cancellation, and observability. The API provides a scope where subtasks are forked; if any subtask fails, the remaining ones are cancelled.

<code>try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Supplier<Object> userInfo = scope.fork(UnstructuredConcurrentDemo::queryUserInfo);
    Supplier<Object> stock = scope.fork(UnstructuredConcurrentDemo::queryStock);
    scope.join(); // wait for all or failure
    System.out.printf("Result: userInfo=%s, stock=%s%n", userInfo.get(), stock.get());
}
</code>

If a subtask throws an exception, other subtasks are cancelled, preventing resource leaks.

Virtual threads dramatically improve throughput when the number of concurrent tasks is high and the workload is not CPU‑bound.