Semantic Search Recall Techniques at JD: Dual‑Tower Model, Graph Model, Synonym Recall, and Joint Index Training

JD's search recall process consists of four stages—recall, coarse ranking, fine ranking, and re‑ranking—where recall draws from both inverted indexing and semantic retrieval. Traditional inverted indexing struggles with semantic similarity, and conventional synonym methods are labor‑intensive, prompting a shift to deep‑learning‑driven solutions.

1. Dual‑Tower Semantic Recall Model

The dual‑tower architecture embeds queries and items into a shared low‑dimensional space, using n‑gram tokenization for queries and rich item features (title, brand, category, delivery method) for items. Query and item towers share an embedding matrix at the input layer, and their outputs are combined via dot‑product. Offline, item embeddings are exported to build an index; online, a query embedding is generated and used to retrieve candidates from the index. To reduce latency, query and index services are merged into a single endpoint.

To address query ambiguity (e.g., "Apple" meaning both the brand and the fruit), the model was enhanced with multi‑head attention: multiple query heads are learned, each attended separately, allowing the system to recall both product types.

2. Graph Model (SearchGCN)

A heterogeneous click graph is constructed with nodes for queries, items, shops, and brands. Edges represent user clicks (query‑item) and attribute relationships (item‑brand, item‑shop). Two‑side GCNs aggregate two‑hop neighborhoods for both query‑centric and item‑centric subgraphs, using attention‑based message passing and sum‑fusion of node embeddings.

Embedding visualizations (t‑SNE) show that SearchGCN produces tighter, more category‑aware clusters compared with baseline semantic models, improving recall for low‑frequency items and short queries.

3. Synonym Recall Model

The synonym system uses a two‑stage transformer: a forward model generates candidate titles from a query, and a backward model reconstructs the query from the title. To better align query‑to‑query semantics, the stages are merged into a joint training model that adds a query‑to‑query generation loss.

During inference, the forward model proposes titles, the top‑probability title is selected, and the backward model generates the final query with stochastic decoding to increase diversity.

4. Joint Index Training Model (PQ‑Integrated)

To mitigate the precision loss of Approximate Nearest Neighbor (ANN) search based on Product Quantization (PQ), the PQ process is embedded as a learnable layer within the model. The layer performs rotation (orthogonal matrix), coarse quantization, PQ sub‑space clustering, and inverse rotation, allowing end‑to‑end optimization of sub‑space partitioning and codebooks.

Experiments on JD private data, MovieLens, and Amazon datasets demonstrate consistent improvements in precision@100 and recall@100. The joint model also simplifies deployment: the trained model and its index are exported together, eliminating separate index‑building steps and reducing serving complexity.

Q&A Session

Questions covered orthogonal matrix initialization (identity matrix with steepest block coordinate descent), loss composition for synonym models (forward + backward + query‑to‑query loss), business integration across JD retail channels, and implementation details of multi‑head projection matrices.

The presentation concluded with thanks to the audience.