Practical Experience of JD E‑commerce Recommendation System: Architecture, Ranking, Real‑time Updates, and Experiment Platform

Today we share JD e‑commerce recommendation system practice, mainly including an introduction, sorting module, real‑time updates, recall and first‑stage ranking, and the experiment platform.

Introduction

The classic recommendation method is collaborative filtering, which includes user‑based and item‑based approaches, but most modern systems use a learning‑to‑rank framework.

The recommendation system can be divided into online and offline parts. The online part assembles results for each user request (recall, ranking, post‑processing). The offline part performs data analysis on user logs and feeds results back to the online system.

Sorting Module

1. Model Structure

Deep learning ranking models typically use large embedding tables for IDs, requiring distributed training and a deployment pipeline. Feature engineering remains essential for performance gains.

2. Practice

Model training platform and ranking service are critical; TensorFlow Serving is commonly used to deploy deep models efficiently.

Feature logs ensure online‑offline consistency, enabling real‑time feedback to be incorporated into training samples.

3. Real‑time Update

Features and models are updated in real time to capture user interest shifts, promotional events, and fast‑changing item attributes. Real‑time signals such as click‑through rate and order data are incorporated, while model updates may rely on conservative strategies due to order latency.

Thoughts

Sorting alone is insufficient; challenges include result similarity, multiple optimization goals, and limited computation resources.

Result similarity harms user experience.

Balancing click‑through, conversion, dwell time, etc., is required.

Unlimited computation would allow exhaustive sorting, which is unrealistic.

1. Diversity

To avoid homogeneous results, a greedy diversity ranking algorithm selects items with the highest novelty score based on KL‑divergence of product‑term distributions.

2. Multi‑objective

Multiple objectives (clicks, conversion, duration) are handled via multi‑model ranking and adjustable weighting.

3. Multi‑stage Ranking

Due to resource constraints, a funnel‑like multi‑stage ranking uses increasingly complex models on progressively smaller candidate sets.

Index‑Based First‑Stage Ranking

Inverted indexes enable fast document retrieval. Various index‑based recall methods (item‑CF keys, product terms, hot categories, promotions) are used, followed by quality‑score truncation and linear/GBDT models.

The weak‑and (wand) operator provides a middle ground between AND and OR, allowing upper‑bound pruning for efficient linear model ranking.

KNN Recall

Embedding‑based KNN recall finds the k nearest candidates to a user embedding. Approximate nearest neighbor search (e.g., FAISS) uses hierarchical quantization (Lloyd clustering, two‑level indexes) to accelerate retrieval.

Experiment Platform

A layered experiment platform partitions traffic using different hash functions per layer, enabling parallel experiments and rapid feature/model iteration.

Overall, the sharing covered JD's recommendation architecture, ranking strategies, real‑time updates, diversity and multi‑objective considerations, first‑stage index ranking, KNN recall, and a scalable experiment platform.