ROFF: A High‑Performance Seven‑Layer Rust‑Based Gateway with TLS Offload, QUIC/HTTP3, and Dynamic Module System

This article introduces ROFF, a self‑developed high‑performance seven‑layer gateway built by the Xiaohongshu access team using the Rust programming language. It provides high‑throughput load balancing, TLS offload, QUIC/HTTP3 support, and a flexible module plug‑in architecture. The paper describes the system architecture, request processing pipeline, module extensibility, TLS hardware offload, hot‑reload/upgrade mechanisms, and configuration format.

System Overview

ROFF is written in Rust, offering memory safety comparable to C/C++ while consuming very little memory. It has been deployed in Xiaohongshu’s own data centers, handling core traffic without any production crashes. The architecture includes a master process for lifecycle management and a worker process that spawns multiple worker threads to handle requests.

Key Features

Memory Safety & High Performance : Rust’s ownership model guarantees memory safety and zero‑cost abstractions, allowing high‑performance networking code without garbage collection overhead.

Rich Proxy Capabilities : Supports multiple load‑balancing algorithms, active health checks, EDS service discovery, and dynamic configuration changes without restarts.

TLS Hardware Offload : Implements a Keyless TLS offload using Rustls with fallback to OpenSSL, accelerating TLS handshakes via hardware accelerators.

Hot Reload / Hot Upgrade : Uses Unix Domain Sockets (UDS) to transfer file descriptors between old and new processes, ensuring zero‑downtime during upgrades.

Extensible Module System : Provides over 30 built‑in HTTP filters and allows developers to add custom modules via Rust macros. The filter chain follows an onion model similar to Gin/Koa.

Configuration : Uses KDL format for concise, hierarchical configuration, compatible with most Nginx directives.

HTTP/3 & QUIC Support : Integrates nghttp3 via FFI to provide stable HTTP/3 capabilities, as native Pingora support is lacking.

Process Model

ROFF adopts a master‑worker‑thread model. The master process monitors signals (restart, reload, config update). The worker process runs multiple threads: a main thread handles configuration parsing, health checks, and service discovery, while worker threads handle request I/O. This reduces inter‑thread communication overhead and simplifies hot‑reload.

Hot Reload / Upgrade Mechanism

During an upgrade, the master process spawns a new worker process, transfers listening sockets via UDS, and lets the old worker finish existing connections before exiting. This avoids connection drops and reduces latency spikes.

// Example of hot‑reload using Unix Domain Sockets (FD transfer)
if let Some(h) = HANDLE.get() {
    h.spawn(future)
} else {
    log::warn!("spawn task on current thread scheduler, which maybe not your intention");
    tokio::spawn(future)
}

Module Development Example

The following snippet shows how a request filter is defined in Pingora and how ROFF extends it with a next parameter to control filter flow.

// Pingora
async fn request_filter(&self, session: &mut Session, ctx: &mut Self::CTX) -> Result
;

// ROFF
async fn request_filter(&self, session: &mut HttpServerSession, ctx: &mut RequestSession, next: RequestFilterNext<'_>) -> Result
;

Developers can use next#call().await , next#ignore_call(...).await , next#call_to_end(...).await , etc., to customize filter execution order.

Configuration Example (KDL)

master_process true
statsd_endpoint "127.0.0.1:9125"
http {
    proxy_read_timeout "60s"
    ssl_backend "rustls"
    ssl_certificate "../certificates/www.example.org.full.cert.pem"
    ssl_certificate_key "../certificates/www.example.org.key.pem"
    include "./upstream.kdl"
    include "./servers.kdl"
    server {
        listen "tcp://0.0.0.0:443" default-server=true
        listen "tcp://[::]:443" default-server=true
        http2 true
        ssl true
        http3 "on"
        server_name "_"
        location "/" { proxy_pass "http://backend" }
        location "/header" {
            add_header "Remote-Addr" "$remote_addr"
            proxy_set_header "URI" "$request_uri"
            return 200
        }
    }
}

Performance Comparison

Benchmarking against Nginx on identical hardware shows comparable throughput for both HTTP and HTTPS traffic. ROFF achieves similar QPS while offering additional features such as TLS hardware offload and HTTP/3 support.

Future Plans

Gradual rollout of public HTTP/3 support.

Open‑source the project and add more Nginx directives.

Integrate Deno to provide JavaScript‑based extensibility similar to OpenResty.

Continue improving compatibility with Pingora and expanding module APIs.

The article concludes with acknowledgments of the team members responsible for the gateway development.