Pingora: Cloudflare’s High‑Performance Rust‑Based Proxy Replacing NGINX

Cloudflare announced last year that it would deprecate NGINX and adopt its self‑developed next‑generation proxy service Pingora, claiming it is faster, more efficient, and more secure.

Overview

Pingora is a new HTTP proxy built in Rust that processes over one trillion requests per day, delivering higher performance while using only one‑third of the CPU and memory of the previous proxy infrastructure.

Why Build a New Proxy

As the world’s largest free CDN provider, Cloudflare’s edge layer handles the greatest volume of web requests, and the existing NGINX‑based architecture could no longer meet performance, scale, and feature requirements.

Architectural Limits Hurt Performance

NGINX’s worker‑process model creates load‑balancing issues because each request is handled by a single worker, leading to CPU core imbalance and slower speeds.

The model also limits connection reuse: a connection can only be reused within the same worker, and adding more workers disperses connections across isolated pools, reducing reuse efficiency and increasing hardware consumption.

Some Feature Types Are Hard to Add

NGINX, while a solid web server and load balancer, does not easily support many of Cloudflare’s advanced use cases, such as sending retries to different origins with distinct headers, or handling non‑RFC‑compliant HTTP traffic.

NGINX is written in C and lacks memory safety, making it prone to bugs and security issues; extending it often requires risky modifications to the upstream codebase.

Choosing to Build Our Own

Over several years Cloudflare evaluated three options: continue investing in a customized NGINX, migrate to a third‑party proxy like Envoy, or build an internal platform from scratch. After repeated assessments, the team decided that building a bespoke proxy offered the best long‑term ROI.

Pingora Project

Design Decisions

To achieve a proxy capable of handling millions of requests per second, the team chose Rust for its memory‑safety guarantees without sacrificing performance.

Instead of relying on existing HTTP libraries such as hyper, Cloudflare built its own library to maximize flexibility and enable rapid innovation.

The proxy must handle many non‑standard HTTP cases that violate RFCs, a common challenge on the open internet.

Support for extended status codes (e.g., 599‑999) was added after extensive discussion with the hyper community.

A multithreaded scheduler with work‑stealing, powered by the Tokio async runtime, was selected to share resources like connection pools efficiently.

The design includes a programmable request‑lifecycle interface similar to NGINX/OpenResty, allowing developers to write request‑filter code that can modify or reject requests.

Pingora Is Faster in Production

Performance data shows median TTFB reduced by 5 ms and the 95th percentile by 80 ms compared with the legacy service.

These gains stem from a shared‑across‑threads connection pool that improves reuse and reduces TCP/TLS handshake overhead.

New connections per second are only one‑third of the previous service, and connection‑reuse rates rose from 87.1 % to 99.92 %, saving roughly 434 years of handshake time per day.

More Features

The developer‑friendly interface removes previous limitations, enabling rapid addition of core features such as new protocols.

For example, HTTP/2 upstream support was added with minimal effort, paving the way for gRPC delivery to customers.

Pingora also introduced Cache Reserve, using Cloudflare R2 storage as a caching layer, unlocking previously impossible product capabilities.

Higher Efficiency

Under the same traffic load, Pingora consumes about 70 % less CPU and 67 % less memory than the old service.

Rust code outperforms the previous Lua implementation, and the multithreaded model reduces lock contention and eliminates unnecessary TLS handshakes.

Higher Security

Rust’s memory‑safety semantics protect against undefined behavior, allowing engineers to focus on feature development rather than debugging memory errors.

Since its launch, Pingora has processed trillions of requests without a single crash caused by its own code.

Summary

In summary, Cloudflare has deployed a faster, more efficient, and more versatile internal proxy—Pingora—as the foundation for current and future products, with plans to open‑source it after further maturation.