Advances in Search Advertising Models with Large Language Models (2024)

With the emergence of the large‑model era, Alibaba Mama investigates whether search‑promotion models can evolve similarly to the deep‑learning boom, outlining the 2024 thinking and practice.

The evolution is described along three “lines”: explicit (inductive bias design), implicit (exponential compute growth), and auxiliary (advances in CV and NLP), which together form the core scaling‑law roadmap for search advertising.

A three‑stage paradigm—Pre‑train, Post‑train, and CTR—drives both perception (multimodal embedding, MIM) and reasoning (user‑behavior large model, LUM), shifting deeper model scaling from the downstream task to the upstream stages.

Large models reshape the advertising system by leveraging pre‑training and post‑training to unify the full‑stack pipeline, achieving notable gains in rewrite, recall, relevance, and creative modules.

Explicit line focuses on inductive bias, such as designing architectures that embed prior assumptions (e.g., CNN’s translation invariance) to improve generalization for sparse ID features.

Implicit line emphasizes compute scaling; despite Moore’s law, memory bandwidth remains a bottleneck, prompting joint algorithm‑engine optimizations to maximize GPU utilization for sparse embeddings.

Auxiliary line draws inspiration from CV/NLP breakthroughs (AlexNet, Word2Vec, Transformers) to accelerate advertising model innovation, including multimodal backbones like BEiT3, BGE, BLIP2, EVA2.

The LUM model treats user‑behavior sequences as a next‑item prediction task, exploring tokenization strategies and a hierarchical architecture that separates semantic encoding from collaborative filtering.

Future work highlights the need for high‑performance inference architectures to bring large models into real‑time online serving, turning LLMs from auxiliary enhancers into core components of the search‑advertising stack.