Xiaohongshu Recommendation Engineering Architecture: Graph Architecture, Hot Deployment, and Practices

With the rapid growth of mobile internet, personalized recommendation has become essential for user experience; Xiaohongshu, a lifestyle platform for young users, faces massive user‑generated content and complex business logic, prompting a redesign of its recommendation system architecture.

1. Business Overview – The recommendation system now handles diverse content types (images, videos, products, live streams, comments) and a multitude of algorithmic strategies, requiring modular data loading, multi‑path recall, extensive feature engineering, and multi‑stage ranking pipelines that process millions of candidates.

Challenges include massive distribution pools, deep feature engineering, and multi‑layer experimental setups, demanding higher reusability, maintainability, and scalability.

2. Graph Architecture Practice – Xiaohongshu built a unified recommendation infrastructure on hybrid cloud, abstracting core services into data platforms, engine layers (inverted index, vector recall, feature and ranking engines), and a Java‑based Ark graph‑computing framework. Ark provides an API layer for traffic control and a container layer with operators and datasets, enabling rapid assembly of new recommendation scenarios.

The framework encapsulates parallel processing, dynamic routing, and sub‑graph nesting, allowing developers to focus on custom operators while the platform handles common functionalities.

3. Hot Deployment for Algorithm Efficiency – To shorten the code‑to‑production cycle, Ark employs isolated class loaders and Spring contexts for version isolation, enabling plug‑in based extensions and AB‑routing for seamless traffic shifting. Hot deployment avoids full service restarts, but introduces challenges such as memory leaks, resource cleanup, class conflicts, and JIT‑induced jitter, which are mitigated through unified thread‑pool management and careful plugin design.

Future directions include extending hot deployment to production for faster rollbacks, moving toward serverless architectures to reduce operational costs, and implementing elastic scaling to handle traffic spikes, especially for live‑stream recommendation scenarios.

4. Q&A – Discussed issues such as class‑loader memory overhead, serialization choices (Thrift vs. Protobuf), and strategies for handling sudden traffic surges through dynamic degradation and fast‑path pipelines.

Overall, the redesign demonstrates how a large‑scale recommendation system can achieve higher efficiency, scalability, and agility through graph‑based architecture and hot‑deployment techniques.