Tengine-Ingress: High‑Performance Cloud‑Native Ingress Gateway for Alibaba Group

Tengine serves as Alibaba Group's 7‑layer application traffic entry gateway. Its core engine, Tengine, is a high‑performance HTTP reverse‑proxy that handles tens of millions of QPS for large‑scale events such as Double‑11.

Although Tengine is known for performance and availability, the legacy Unified Access (统一接入) suffers from several limitations: domain name additions and updates are not applied dynamically, new certificates cannot be hot‑reloaded, HTTP(S) configurability and observability are weak, routing capabilities are limited, and cluster‑wide changes have excessive impact.

With the standardization of cloud‑native Ingress gateways and the rollout of Alibaba Cloud Serverless Infrastructure (ASI), a new application called AServer‑Ingress, built on the Kubernetes Ingress standard, upgrades the Unified Access layer. Its technical foundation, Tengine‑Ingress, deeply optimizes the upstream ingress‑nginx codebase and integrates Tengine‑Proxy, delivering higher performance, availability, and full resolution of the previous pain points.

Key components :

Tengine‑Ingress Controller – an Ingress controller based on Tengine‑Proxy that extends the native Ingress spec with Server, Backend, TLS, Location, and Canary fields.

Tengine‑Proxy – subscribes to shared‑memory updates from the controller and routes external traffic to services inside the K8s cluster.

Major innovations :

Dynamic, lossless configuration: domain names, routing rules, and certificates take effect instantly without reloading Tengine, preserving long‑lived connections.

Granular gray‑rollout: per‑domain and per‑certificate batch rollout controlled by custom annotations (e.g., nginx.ingress.kubernetes.io/ingress-rollout ).

Dual‑certificate TLS support: ECC and RSA certificates can be loaded simultaneously, with future extension to SM2/SM3.

Observability per domain: real‑time QPS, success rate, latency, and payload size metrics.

Global consistency check: an MD5 checksum of all Ingress objects is stored in a custom CRD (IngressCheckSum). Updates are applied only when the checksum matches, preventing dirty data from being cached.

Example of a TLS configuration snippet:

tls:
- hosts:
  - {host1}
  secretName: {secret name1}
- hosts:
  - {host1}
  secretName: {secret name2}

Definition of the checksum CRD (Go structs):

type IngressCheckSum struct {
  metav1.ObjectMeta
  Spec IngressCheckSumSpec
}

type IngressCheckSumList struct {
  metav1.TypeMeta
  metav1.ListMeta
  Items []IngressCheckSum
}

type IngressCheckSumSpec struct {
  // Timestamp is the time when the md5 of all the ingress was calculated.
  Timestamp metav1.Time
  // Checksum is the md5 of all the ingress.
  Checksum string
  // Ids describes which id will match this ingress.
  Ids []string
}

Performance evaluation :

In HTTPS short‑connection tests (TLS 1.2, ECDHE‑RSA‑AES128‑GCM‑SHA256, 22 KB response), Tengine‑Ingress reduced average response time (RT) by ~9 ms compared with the original Tengine, a 20.7 % improvement. CPU consumption dropped from 81.9 % to 67.3 % (≈20 % gain). Memory usage fell from 20.4 % to 8.8 % because workers share a single dynamic configuration instead of loading static copies.

Scalability tests with 20 000 Ingress objects and 10 QPS update rate showed that watch latency remains bounded even when the number of Tengine‑Ingress pods scales to thousands.

Future direction :

Tengine‑Ingress will evolve into a universal API gateway, adding support for 4/7‑layer TLS, TCP, UDP, and gRPC routing, and integrating XQUIC for QUIC/HTTP3 and XUDP for high‑performance user‑space UDP forwarding. The roadmap also includes upgrading the NGINX core, supporting ECC + RSA + SM2 certificates, and enhancing distributed service discovery.

All source code and related projects are open‑source:

Tengine: https://github.com/alibaba/tengine

Tengine‑Ingress: https://github.com/alibaba/tengine-ingress

XQUIC: https://github.com/alibaba/xquic

XUDP: https://gitee.com/anolis/libxudp