EdgeRec: Leveraging Edge Computing for Real‑Time Recommendation Systems

Recommendation systems are critical to modern internet products, but traditional cloud‑centric pipelines suffer from latency, bandwidth constraints, and scalability issues when serving billions of users. Edge computing offers lower latency, reduced bandwidth usage, enhanced privacy, and higher stability, making it an attractive alternative for real‑time personalization.

The EdgeRec framework rethinks the entire recommendation workflow by moving the final decision‑making stage to the user device. It introduces a two‑stage on‑device perception module that continuously ingests fine‑grained user actions (exposures, clicks, dwell time) and encodes heterogeneous <item, action> pairs into separate streams for exposure and click behaviors. These streams are fused with item features to produce a real‑time user state that can be queried by downstream ranking models.

For on‑device reranking, EdgeRec proposes the Context‑aware Reranking with Behavior Attention Networks (CRBAN). CRBAN treats candidate items as queries, the user’s behavior sequence as keys, and the fused item‑action embeddings as values, allowing the model to attend to the most relevant past interactions when scoring each candidate. Offline experiments on CIKM‑2020 data show substantial gains over baseline DNN rerankers.

The system design decouples the heavy embedding matrix from the on‑device inference engine: embeddings are stored in the cloud and fetched on‑demand, while the lightweight ranking network runs locally using the MNN inference engine. This hybrid architecture preserves the benefits of cloud‑scale models while enabling low‑latency, privacy‑preserving inference.

EdgeRec also addresses the mixing problem in information‑flow feeds, where heterogeneous cards (videos, ads, products) must be ordered jointly. A two‑phase mixing pipeline first estimates click‑through rates for each candidate using the Edge Surrounding‑Aware Network (ESAN), which incorporates both user behavior and surrounding card context. The second phase solves a constrained integer‑programming problem (dynamic knapsack) to select positions that satisfy exposure‑share and dispersion rules, achieving up to 7% CTR improvement in short‑video streams.

To further personalize recommendations, EdgeRec explores on‑device training for “thousand‑users‑one‑model” (千人千模). A shared global model is fine‑tuned locally on each user’s sparse data, improving low‑activity user metrics while preserving privacy. Because per‑user data are scarce, a meta‑learning approach based on MAML is employed, and a personalized learning‑rate variant (PAML) mitigates task‑overfitting for long‑tail users. This work has been published at AAAI‑2021.

Overall, EdgeRec demonstrates that moving perception, reranking, mixing, and even model training to the edge can dramatically reduce latency, increase relevance, and boost business KPIs, as evidenced by significant CTR and PV gains across multiple Alibaba services.