Google’s Maglev Load Balancer: Architecture, Performance, and Design Insights

Google’s internal technologies often attract attention, and at the beginning of the year the company published a paper describing the implementation of its internal load balancer, offering a glimpse of what may be the world’s best load‑balancing system.

Typical load balancers must trade flexibility for performance. User‑space solutions such as HAProxy and Nginx are easy to configure but suffer from deep network‑stack processing, while kernel‑space LVS achieves orders‑of‑magnitude higher throughput at the cost of complexity. The article asks what secret recipe Google uses to reach high performance.

For most sites a single load balancer suffices, but Google’s traffic volume requires the load balancer itself to be horizontally scalable and highly available. The article explores how Google accomplishes this.

When a client accesses www.google.com , DNS returns a VIP address nearest to the client, providing horizontal scaling at the DNS layer. The request then reaches a router that uses ECMP to distribute traffic evenly across multiple peer load balancers, achieving another layer of scaling. The load balancer forwards packets to servers in a mode similar to LVS’s DR, rewriting the source address to the client’s address and letting the server’s response be sent directly to the router, bypassing the load balancer and reducing its load.

The load balancer is called Maglev (named after magnetic‑levitation trains). While its high‑level flow resembles LVS’s DR mode, its internal implementation is completely different.

Google considers the Linux kernel a performance bottleneck. Instead of passing packets through the full TCP/IP stack and kernel filters, Maglev is placed directly on the NIC, connecting to the NIC’s input and output queues. It inspects only the first few bytes of each packet to extract the five‑tuple (source IP, source port, destination IP, destination port, protocol), which is sufficient for forwarding, and then pushes the packet straight to the NIC’s output without further processing.

Similar ideas are used by Meituan, which implements a DPDK‑based load balancer on the NIC for several‑fold performance gains over LVS. Maglev goes further by writing directly to the NIC without any DPDK‑style wrapper, sharing memory with the NIC to avoid copying packets between the NIC and the balancer. The NIC input queue, the load balancer, and the NIC output queue all share a single memory pool, with three pointers moving packets through the pipeline, achieving extreme efficiency in memory handling.

Additional optimizations include binding each CPU core to a dedicated thread to avoid cache misses and mitigating memory contention. The result is an average packet‑processing latency of about 350 ns; occasional latency spikes up to 50 µs occur when the NIC batches packets or waits for a timer interrupt.

Unlike single‑point load balancers, Maglev operates as a cluster, which raises challenges around stateful TCP connections. Google solves this by employing a custom consistent‑hashing algorithm (Maglev Hashing) that maps the five‑tuple to a fixed backend server, effectively making the stateful service appear stateless and allowing seamless updates, scaling, and server churn.

The paper also shares operational experiences and future directions; interested readers can view the original PDF (http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44824.pdf).

Copyright notice: content sourced from the internet, rights belong to the original author. If any infringement is identified, please inform us for removal.