Preference‑Oriented Diversity Model Based on Mutual Information for E‑commerce Search Re‑ranking (SIGIR 2024)

Re‑ranking in e‑commerce search aims to reorder items by considering their relationships, but existing methods often improve scoring accuracy at the expense of result diversity, or vice‑versa. Users exhibit varying diversity needs across decision stages, requiring a balanced approach.

The authors propose PODM‑MI (Preference‑oriented Diversity Model based on Mutual Information), which simultaneously addresses accuracy and diversity. The framework consists of two main components: PON (Preference‑Oriented Modeling) for capturing user diversity preferences, and SAM (Similarity‑aware Mutual‑information) for aligning candidate items with those preferences.

PON models user intent using a multivariate Gaussian distribution derived from historical queries, session information, and behavior streams. The distribution’s mean and diagonal covariance capture the evolving uncertainty of user preferences, while a parallel Gaussian models item‑level diversity.

SAM quantifies the alignment between user preference and item diversity via mutual information. Because direct estimation is intractable, a variational lower‑bound is derived (see ), enabling gradient‑based optimization. An learnable utility matrix further adjusts the influence of each item according to the enhanced consistency.

The overall loss combines a PRM classification loss (see ) with a mutual‑information loss, encouraging both accurate ranking and diverse outcomes.

Extensive online A/B testing on JD.com’s main search engine demonstrates that PODM‑MI significantly improves user conversion probability (UCVR) and result diversity. Visual analysis using entropy measures and T‑SNE clustering shows clear separation of user intent groups, confirming the model’s ability to adaptively balance relevance and variety.

Case studies illustrate how the model yields more diverse results for users with broad query histories (e.g., "Switch, Zelda, phone case…") and more focused results when intent becomes specific (e.g., repeated searches for "dress").

Future work includes incorporating finer‑grained features for better intent modeling, refining the update mechanism for user preference representations, and making intent modeling explicitly influence ranking decisions.

The research team also announces open recruitment for positions related to large‑model‑driven retrieval and ranking, inviting interested candidates to apply.