Real-Time Controllable Multi-Objective Re‑ranking for Taobao Feed

The presentation introduces the challenges faced by Taobao's information‑flow recommendation, such as content diversity, cold‑start exposure, mixed media types, multi‑supply fusion, and numerous business objectives, and explains why traditional pipeline‑style ranking struggles to satisfy these requirements.

It then outlines three re‑ranking modeling paradigms: V1 (context‑aware single‑point scoring), V2 (sequential item selection with state‑based attention), and V3 (reward‑driven reinforcement learning that eliminates the need for explicit labels). Each paradigm’s strengths and limitations are discussed.

The core solution is an actor‑critic architecture where the actor (a DeepSet encoder plus PointerNet decoder) generates a ranked sequence and the evaluator estimates a reward based on multiple utilities (click‑through rate, diversity, freshness, etc.). The reward is a weighted sum of utilities, and the weights \(w\) can be specified in real time.

To enable dynamic weight adjustment without retraining, a hypernetwork predicts weight‑sensitive parameters \(\theta_w\) for the re‑ranking model. During offline training, each sample randomly draws a weight vector from a predefined distribution; the actor generates a sequence, the evaluator computes the reward, and gradients update both the hypernetwork and the weight‑insensitive parts of the model.

Extensive online experiments (A/B tests) demonstrate that the controllable re‑ranking model improves click‑through rate, cold‑start content ratio, shop diversity, group ordering, user dwell time, and content volume compared with the baseline pipeline approach.

The article concludes with a Q&A covering topics such as weight personalization, offline training with sampled weights, handling of mixed media, latency (≈20 ms P99), feature engineering for heterogeneous items, and evaluation metrics (reward and better‑percentage).