Metis: Understanding and Enhancing In-Network Regular Expressions

This paper introduces Metis, an AI-based solution for fast and accurate network packet identification and processing, which eliminates the need for setting individual rules for each scenario like traditional regex matching, providing better generality.

Research Background: TGW (Tencent Gateway) serves as Tencent's public network gateway, handling most of Tencent's public network bandwidth and constantly facing malicious traffic attacks. Current anomaly detection methods include signature-based detection and anomaly-based detection. While regex matching offers high accuracy, it consumes significant computational and storage resources with poor generalization ability. Neural network solutions typically suffer from "cold start" problems and are difficult to deploy on resource-constrained network devices like P4 switches with only tens of MB storage.

Technical Architecture: Metis first converts RE rules into Deterministic Finite Automata (DFA), then trains byte-level RNNs (BRNN) based on these DFAs. The design incorporates existing RE-based expert knowledge into BRNN to solve the "cold start" problem. To adapt to resource-constrained network devices, knowledge distillation (KD) is used to transform BRNN into Pooling Soft Random Forests (PSRFs) models, which inherit BRNN's training effectiveness with significantly reduced resource consumption.

Performance Results: Experiments show that PSRF models achieve 74x higher throughput than regex matching on programmable switches. Metis achieves accuracy comparable to RE even without training data, with accuracy improving as training data increases.

Practical Applications: In CDN access scenarios, Metis can improve DDoS protection system performance by 1.3x, reducing computational resource consumption by 32%. In P4 forwarding scenarios, Metis deployed on P4 devices achieves 69,711,100 pps throughput compared to only 9,450 pps for RE-based methods on 32-core CPU—a 74x improvement.