Tencent's Wuliang Deep Learning System for Large‑Scale Recommendation: Architecture, Challenges, and Solutions

Guest Speaker: Yuan Yi, Ph.D., Tencent Expert Researcher (organized by DataFunTalk).

Overview: With the rapid evolution of recommendation technology, real‑time data processing, massive user traffic, and diverse optimization goals create significant challenges for training and serving high‑dimensional models. This talk focuses on Tencent's Wuliang system applied to recommendation workloads.

1. Background and Problems

The Wuliang system addresses the need for real‑time data handling, dynamic modeling objectives, and massive scale in recommendation scenarios, which differ from traditional CV/NLP tasks.

2. AI Full‑Process Relationship

Recommendation pipelines involve user behavior collection, sample generation, model training, and online serving, requiring tighter latency and data freshness than content‑understanding pipelines.

Data Real‑Time: User actions must be reflected instantly in model updates.

Dynamic Modeling Goals: Models must adapt to changing user interests.

3. Technical Bottlenecks in Recommendation

Huge data volume (billions of exposures, billions of clicks, massive DAU).

Strong user interaction relationships demanding higher timeliness.

Need for trillion‑parameter model training and terabyte‑scale model serving.

Requirement for high‑throughput, low‑latency online services.

4. Wuliang Solutions

4.1 Computing Framework

Wuliang integrates a parameter‑server architecture with TensorFlow, enabling on‑demand parameter fetching and efficient gradient updates. It employs parameter compression, zero‑copy communication, and lock‑free parallel updates on the server side.

4.2 Model Computation

Large sparse embeddings are handled by fetching only active keys per batch, reducing memory usage. The system bridges parameter‑server operations with TensorFlow's graph construction and automatic differentiation.

4.3 Inference Service

Multi‑replica, in‑memory serving ensures strong consistency when needed. A distributed serving cluster uses a three‑level cache (SSD → memory → GPU) to store low‑, medium‑, and high‑frequency parameters, achieving sub‑10 ms latency.

4.4 Continuous Online Pipeline

Models are sliced into DNN and sparse embedding parts, deployed to different nodes, and updated via full, incremental, or real‑time pipelines (TB, GB, or KB updates respectively).

4.5 Model Compression

Variable‑Length Embeddings: Reduce values for low‑frequency features.

Group Lasso (key reduction): Apply L21 regularization to prune sparse keys.

Mixed‑Precision: Use float16/int8/int4 representations.

Quantization: 1‑bit or 2‑bit compression with optional quantization‑aware training.

5. System Evolution

Scaling out (more nodes) and scaling up (faster single‑node training) are combined with multi‑level caching to handle TB‑scale models on GPUs, leveraging SSD for parameter staging and reducing cross‑node communication.

6. Q&A Highlights

Inference replicas store parameters in memory with optional strong synchronization.

Three‑value quantization keeps full‑precision models during training and uses quantized inference.

Zero‑copy communication avoids data movement by pre‑ordering keys based on their server locations.

Model rollback is the fallback when training data is polluted.

Feature engineering can be performed either upstream or via a plugin in the inference service.

Thank you for attending the session.