Bayesian Hierarchical Calibration for Online Advertising Scoring

Accurate scoring in online advertising is crucial for both client performance and platform revenue. The article distinguishes between pre‑click (pCTR) and post‑click (pCVR) scoring, explaining their roles in ad ranking and bidding decisions.

The current scoring models, primarily deep learning based, face several challenges: lack of value accuracy, cold‑start problems, large dimension‑wise effect differences, and sample skew.

Calibration is presented as a direct solution. A lightweight, interpretable calibration module should be simple, explainable, and robust.

Problem Analysis

Statistical tests reveal significant gaps between estimated and actual CTR across various dimensions, indicating value‑accuracy issues that calibration must address.

Modeling Scheme

Four hierarchical modeling approaches are described:

Global dimension modeling: a single bias for all samples.

Independent dimension modeling: separate biases per customer, sharing a common prior, forming a Bayesian hierarchical model.

Cross‑dimension modeling: adds interaction terms between customer and traffic channel to capture non‑orthogonal effects.

Live‑ad scenario modeling: combines customer, channel, and score‑level dimensions into a compact model with eight effect terms.

Each approach balances expressiveness, data sparsity, and computational cost.

Inference & Calibration Process

The Bayesian Generalized Linear Model (BGLM) framework uses mean‑field variational inference to approximate posterior distributions of model parameters. The posterior predictive distribution provides a Gaussian approximation for the calibrated score, from which a point decision (e.g., posterior mean) can be derived.

The end‑to‑end calibration workflow includes: constructing training samples, performing variational inference with BGLM, predicting posterior distributions for online samples, and making calibrated scoring decisions.

Risk‑Aware Decision

A risk‑aware objective combines expected posterior density (entropy) and squared calibration deviation, weighted by a factor λ. The optimal calibrated score minimizes this combined risk, reducing over‑correction while accounting for uncertainty.

Evaluation Metrics

Key metrics for calibration effectiveness are AUC (ranking quality), GAUC (grouped AUC for channel‑wise evaluation), PCOC (ratio of observed positive rate to predicted mean), and dimension‑wise error measures (ERROR_dim and ERROR_dim_value).

Empirical results on live‑ad traffic show up to 66% reduction in calibration error for pre‑click scores, 5% revenue lift, and 2‑19% cost reductions for post‑click conversions. Risk‑aware calibration further improves stability.

Conclusion

Bayesian hierarchical calibration offers a simple, lightweight, and interpretable solution for value‑accuracy problems in advertising scoring, especially in live‑ad scenarios. The posterior distribution naturally supports risk‑aware decisions, delivering measurable performance gains.