Understanding Backpressure in Asynchronous Systems

Asynchronous programming has become popular across many languages (Python, Rust, Go, Node, .NET). While async simplifies waiting for long‑running operations, it also introduces backpressure problems that only become apparent when a system is overloaded.

> What is backpressure

Backpressure is the resistance that prevents data from flowing freely through a system. Like a clogged pipe at an airport luggage conveyor, when the downstream container is full, new items cannot be accepted, forcing the system to either wait (queue/buffer) or drop data.

Communicating backpressure is essential; without it, overloaded services may continue accepting work, eventually exhausting resources and causing failures, as illustrated by the Heathrow Terminal 5 baggage handling crisis.

> A bad default example

The following Python asyncio snippet shows a simple echo server that lacks proper backpressure handling:

from asyncio import start_server, run

async def on_client_connected(reader, writer):
    while True:
        data = await reader.readline()
        if not data:
            break
        writer.write(data)

async def server():
    srv = await start_server(on_client_connected, '127.0.0.1', 8888)
    async with srv:
        await srv.serve_forever()

run(server())

In async code, writer.write writes to a non‑blocking socket buffer. If the buffer is full, the write cannot block; the only options are to buffer more data or drop it. Python chooses buffering, which can grow without bound.

To avoid uncontrolled buffering, the asyncio API requires an explicit await writer.drain() after a write:

writer.write(data)
await writer.drain()

TCP already provides hidden flow control, but it is not exposed by the socket API, so developers must add their own mechanisms.

> Queueing vs. backpressure

Using a semaphore to limit concurrent database connections is a common pattern:

from asyncio.sync import Semaphore
semaphore = Semaphore(200)

async def handle_request(request):
    await semaphore.acquire()
    try:
        return generate_response(request)
    finally:
        semaphore.release()

However, this merely queues requests; when the system is heavily overloaded, everyone ends up waiting and eventually the server may run out of memory.

Adding a readiness check can surface overload early:

from hypothetical_asyncio.sync import Semaphore, Service
semaphore = Semaphore(200)

class RequestHandlerService(Service):
    async def handle(self, request):
        await semaphore.acquire()
        try:
            return generate_response(request)
        finally:
            semaphore.release()

    @property
    def is_ready(self):
        return semaphore.tokens_available()

Usage:

response = await handle_request(request)

or with readiness:

request_handler = RequestHandlerService()
if not request_handler.is_ready:
    response = Response(status_code=503)
else:
    response = await request_handler.handle(request)

Libraries such as Trio provide a CapacityLimiter that exposes internal counters and avoids common semaphore pitfalls.

> Data‑flow protocols

For request/response protocols, an overloaded server can return HTTP 503 with a Retry-After header. Persistent‑connection protocols (e.g., HTTP/2) need their own flow‑control frames because TCP’s built‑in control is hidden from the application.

Designing custom data‑flow protocols requires a bidirectional channel so the sender can be told when to slow down.

> Final recommendation

Async/await makes it easy to write highly concurrent code, but without explicit backpressure mechanisms the system can quickly become unstable. Library authors should document and expose backpressure and flow‑control features prominently in their APIs.