Evolution of Search EE System: Adaptive Exploration, Scenario Modeling, End-to-End Scoring Consistency, and Context-Aware Brand Store Detection
This article outlines the recent full‑cycle iterations of JD’s search Explore‑Exploit (EE) system, covering adaptive dynamic detection models, upgraded scenario modeling, two‑stage scoring and insertion consistency, end‑to‑end dynamic insertion, and context‑aware brand‑store dimension detection, with detailed methodology, experiments, and online results.
The search EE (Explore‑Exploit) system at JD faces a head‑effect where high‑quality mid‑ and long‑tail items receive limited exposure; breaking this Matthew effect is crucial for e‑commerce platforms.
Stage 1 – Adaptive Dynamic Detection Model : Introduces an Explore‑Net that explicitly models user "browse" vs. "buy" preferences, incorporates browsing‑depth regression as an auxiliary task, and fuses Explore‑Net with the original Exploit‑Net via a dual‑tower architecture. This enables differentiated exploration intensity based on user intent and improves feature utilization.
Stage 2 – Scenario Modeling Upgrade : Shifts from a single‑task click model to a multi‑task framework (CTR + CTCVR) using a lightweight share‑bottom network, allowing richer representation of conversion signals while keeping online latency low. Various fusion strategies were evaluated, with weighted‑sum selected for best offline and online performance.
Stage 3 – Scoring and Insertion End‑to‑End Consistency : Aligns the EE scoring stage with the dynamic insertion stage by mapping predicted browsing depth to the number of inserted items, ensuring that higher exploration scores translate to more insertion slots. The revised pipeline reduces inconsistency between scoring and placement.
Stage 4 – Context‑Aware Brand/Store Dimension Detection : Adds a context perception module that identifies brand or store queries and evaluates the distribution of top‑k results. When head‑brand/store concentration exceeds a threshold, the system disables insertion of similar items, thereby diversifying results.
Experimental Results : The Explore‑Net achieves RMSLE 0.0903 vs. 0.2053 for the baseline XGB model. Offline analyses show increased detection strength with deeper sessions, more balanced score distributions, and higher insertion counts for exploratory users. Online A/B tests report ~0.5% lift in liquidity and exploration success rate while maintaining search efficiency.
Conclusion & Outlook : The iterative upgrades demonstrate that adaptive exploration, multi‑task scenario modeling, end‑to‑end consistency, and context‑aware detection collectively enhance result richness and mitigate the head‑effect. Future work includes expanding training data, extending EE across the full ranking pipeline, and improving product representation for mid‑ and long‑tail items.
DataFunTalk
Dedicated to sharing and discussing big data and AI technology applications, aiming to empower a million data scientists. Regularly hosts live tech talks and curates articles on big data, recommendation/search algorithms, advertising algorithms, NLP, intelligent risk control, autonomous driving, and machine learning/deep learning.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.