Advances in Alibaba Search Advertising Estimation: Model Deepening, Interaction, and System Efficiency (2021 Review)

The document presents a comprehensive technical review of the Alibaba Mama Search Advertising estimation platform for 2021, focusing on three major aspects: model deepening, model interaction, and system efficiency.

1. Model Deepening – The high‑water‑mark precision ranking (CTR) model is further optimized at both the Embedding Layer and Hidden Layer. In the Embedding Layer, binary‑code‑based hash embedding (BC) and adaptively‑masked twins‑based layer (AMTL) are introduced to achieve lossless compression of massive sparse features while preserving accuracy. The Hidden Layer explores new growth points beyond user‑behavior modeling, including an adaptive dynamic‑parameter model (AdaptPGM) that generates per‑traffic‑condition parameters, and a pre‑trained graph neural network (PCF‑GNN) for explicit cross‑feature learning.

2. Model Interaction – The multi‑stage cascade architecture (pre‑ranking → ranking → re‑ranking) is examined. For pre‑ranking, a ranking‑distillation‑based pre‑ranking model (RDPR) aligns pre‑ranking scores with downstream rank‑score signals. In ranking, the “oracle” capability is added to predict position bias and external context, enabling tighter coupling between ranking and re‑ranking. Creative selection is also integrated via a cascade architecture that places a dedicated creative‑ranking tower before the ranking model, using an Adaptive DropNet to balance ID and content features.

3. System Efficiency – To break the general‑purpose compute bottleneck, the team adopts high‑performance‑computing (HPC) training with large batch sizes, mixed‑precision communication, and multi‑hash embedding that reduces model size to ~30 GB, allowing full‑model training on a single GPU. Communication‑efficient All‑Reduce variants and fp16 quantization further accelerate training, achieving a 30× speed‑up.

The report also lists several peer‑reviewed papers (CIKM 2021, SIGIR 2021, WWW 2022) that detail the proposed methods, and provides a brief outlook on future research directions.