Hybrid Embedding Architecture for Large‑Scale Sparse CTR Models

Sparse CTR models rely on massive high‑dimensional sparse features, leading to huge embedding matrices that cause severe storage and communication overhead. Traditional TensorFlow variables and simple parameter servers struggle with static shapes, feature conflicts, and scaling issues.

The Ant AI Infra team introduced a Hybrid Embedding architecture that combines a multi‑layer storage hierarchy with an optimized KV‑Variable memory parameter server, leveraging NVMe SSDs, an Embedding Service, and dynamic hot‑cold feature partitioning based on LFU frequency.

Key components:

KV‑Variable memory parameter server with sharding for training and replica‑based storage for inference.

Embedding Service that provides a shared, high‑scalability service to eliminate redundant embedding copies across tasks.

Hierarchical storage using fast NVMe SSDs for hot embeddings and slower disks for cold embeddings, guided by feature access frequency.

Dynamic cache management and LFU‑based hot‑cold feature division, with lazy‑load indexing and periodic incremental updates.

Technical choices favored a custom high‑performance SSD engine (PHStore) over conventional LSM‑tree stores, achieving superior random read/write performance and linear scalability.

Extensive optimizations—including one‑pass embedding queries, dynamic shard expansion, and fault‑tolerant mixed storage—resulted in roughly 50% memory savings on parameter‑server nodes while maintaining performance comparable to pure DRAM storage.

The Hybrid Embedding solution is now fully deployed in Ant's online learning recommendation pipeline and is slated for open‑source release alongside the DLRover project.