Metis: AI‑Driven In‑Network Regular Expression Enhancement for High‑Performance Traffic Inspection

Recently, the paper "Metis: Understanding and Enhancing In‑Network Regular Expressions"—co‑authored by Tencent TGW gateway team, Tencent DDoS protection team, and Tsinghua University—was accepted at NeurIPS 2023 and received strong praise from both academia and industry.

Metis provides an AI approach that enables rapid and accurate packet identification without the need to craft individual regular‑expression rules for each scenario, improving generality. By applying knowledge extraction, the model is compressed and successfully deployed on Tencent's TGW gateway, a resource‑constrained network device.

The TGW gateway handles the majority of Tencent's public‑facing traffic and must constantly defend against malicious flows. Traditional detection relies on either signature‑based matching (regular expressions) or anomaly detection, both of which have drawbacks such as high resource consumption and poor generalization.

Metis converts regular‑expression rules into deterministic finite automata (DFA) and trains a byte‑level RNN (BRNN) that incorporates expert RE knowledge, mitigating the cold‑start problem of conventional neural networks. To fit the limited resources of network devices, knowledge distillation transforms the BRNN into a Pooling Soft Random Forest (PSRF) model, retaining classification performance while drastically reducing resource usage.

Experimental results show that the PSRF model on programmable switches achieves up to 74× higher throughput compared with traditional RE matching. In accuracy tests, Metis matches RE performance without training data and surpasses it as more labeled data become available.

Metis has already demonstrated industry impact: it can replace RE engines in Tencent's DDoS protection system, cutting processing overhead from 60% to 28% and saving roughly 32% (≈0.64 w) of CPU cores. Deployed on P4‑based Tofino switches, Metis delivers 6.97 Mpps versus 9.45 Kpps for RE‑based methods—a 74‑fold increase.

Beyond DDoS protection, Metis is applied in CDN ingress scenarios and next‑generation TGW architectures that leverage P4 programmable data planes, offering substantial cost reductions (≈60% of previous hardware) and enabling real‑time intrusion detection without off‑loading traffic.

Overall, Metis showcases how AI can enhance in‑network security functions, providing a scalable, high‑performance alternative to traditional regular‑expression matching.