A Comprehensive Overview of Common CTR Prediction Models and Their Evolution

Background

Click‑through‑rate (CTR) estimation is a core technology in recommendation, search, and advertising. This article reviews common CTR prediction models, summarizing their structures, strengths, weaknesses, and relationships.

Model Taxonomy

Distributed Linear Models Logistic Regression (LR) : Simple, low‑complexity, scalable linear model; relies heavily on manual feature engineering and cannot learn unseen feature interactions.

Automated Feature Engineering GBDT+LR (2014) : Uses Gradient Boosted Decision Trees to automatically generate feature vectors, followed by a separate LR stage; non‑end‑to‑end and limited with high‑dimensional sparse features.

Factorization‑Machine (FM) Models and Variants FM (2010) : Learns second‑order feature interactions via latent vectors; handles sparsity and generalizes to unseen interactions. FFM : Introduces field‑aware embeddings for each feature‑field pair, improving expressiveness at higher computational cost. AFM : Adds an attention mechanism to weight second‑order interactions, enhancing interpretability.

Embedding+MLP Shallow Modifications FNN : Pre‑trains FM embeddings and feeds them to a downstream DNN; two‑stage, non‑online‑learning. PNN : Incorporates a Product Layer (inner or outer product) to capture richer feature interactions. NFM : Uses a Bi‑Interaction Pooling layer for second‑order interactions, combined with DNN. ONN : Operation‑aware embeddings assign separate embeddings for different product operations, extending FFM ideas.

Dual‑Tower Model Combinations Wide&Deep (WDL, 2016) : Parallel wide linear model and deep neural network to capture memorization and generalization. DeepFM (2017) : Shares embeddings between FM (low‑order) and DNN (high‑order) components for end‑to‑end learning.

Explicit Feature‑Cross Networks Deep&Cross (DCN, 2017) : Adds a CrossNet that explicitly computes bounded‑degree feature crosses with residual connections. xDeepFM (2018) : Introduces a Compressed Interaction Network (CIN) that learns explicit high‑order, vector‑wise interactions. AutoInt (2019) : Applies multi‑head self‑attention (QKV) to learn explicit feature interactions with interpretability.

Comparative Summary

The progression shows a shift from manual feature engineering (LR) toward end‑to‑end deep models that automatically learn both low‑order and high‑order interactions. Innovations include multi‑embedding strategies (FFM, ONN), attention mechanisms (AFM, AutoInt), and explicit cross networks (DCN, CIN) that improve expressiveness while balancing computational cost.

Conclusion

CTR prediction models have evolved from simple linear classifiers to sophisticated architectures that combine embedding layers, deep neural networks, attention, and explicit cross networks, continually enhancing the ability to capture complex user behavior patterns.