ZEUS: A Self‑Supervised Multi‑Scenario Query Ranking Model for E‑commerce Search

This article introduces the CIKM 2021 paper "Self‑Supervised Learning on Users' Spontaneous Behaviors for Multi‑Scenario Ranking in E‑commerce," which proposes the ZEUS (Zoo of ranking modEls for mUltiple Scenarios) model to address feedback‑closure, data scarcity in small or new scenarios, and joint learning across multiple ranking tasks.

Background : E‑commerce search drives a large share of transactions. Query recommendation (both user‑typed and system‑suggested) is crucial for user growth, with query recommendation accounting for over 20% of traffic. Improving the efficiency of multi‑scenario query ranking directly boosts UV, clicks, and GMV.

Challenges : (1) Feedback‑closure where implicit feedback reinforces popular items and harms long‑tail exposure; (2) Insufficient training data for small or new scenarios; (3) Need for joint learning across scenarios.

Proposed Solution : ZEUS adopts self‑supervised learning. In a pre‑training stage, a Next‑Query Prediction (NQP) task predicts the next user‑typed query from historical behavior, using a Transformer‑based Sequential Interest Model. Hard negative samples are taken from exposed but unclicked queries. The loss combines InfoNCE contrastive loss with cross‑entropy approximations to handle the massive query vocabulary.

Model Architecture : User profile, query sequence, click sequence, dense features, and scenario identifiers are encoded by separate Transformers. Their outputs are concatenated and fed to an MLP to predict CTR for candidate queries. Only the embedding and Transformer layers are shared between pre‑training and fine‑tuning; the prediction layer is task‑specific.

Fine‑Tuning : First, a multi‑scenario fine‑tuning step jointly trains on implicit feedback from all scenarios to learn shared representations. Then, scenario‑specific fine‑tuning refines each model with its own data. Freezing query and item embeddings during fine‑tuning prevents over‑fitting.

Offline Experiments : Datasets from four query‑recommendation scenarios (home‑page, discovery, alliance app, special‑price) were collected. Baselines include GBDT, FINN, YouTube DNN, and DMT. ZEUS outperformed all baselines, with pre‑training contributing the largest boost. Ablation studies confirmed the importance of both query and click sequences, pre‑training, multi‑scenario fine‑tuning, and scenario‑specific fine‑tuning.

Online Results : ZEUS achieved notable UCTR improvements across all scenarios (e.g., +16.6% over FINN in the home‑page scenario) and increased the number of unique recommended queries, demonstrating effective breaking of the recommendation loop.

Case Study : In a real‑world example, ZEUS recommended the query "欧式针织毛衣" by attending to the user's past queries and clicked items, showing both relevance and interpretability.

Future Work : Explore richer pre‑training tasks and contrastive losses, improve multi‑scenario joint learning, and extend optimization to multiple business objectives such as click‑through, purchase, and GMV.

References : The paper cites works on deep interest networks (DIN, DIEN), transformer‑based ranking (DMT), multi‑scenario models (HMoE, SAML, STAR), query recommendation models (GBDT, MEIRec, FINN), and self‑supervised learning (BERT, GPT, SimSiam, CPC).