AdaCalib: Posterior-Guided Feature-Adaptive Calibration Model for Online Advertising

Accurate probability estimation of user behaviors (click, conversion, dwell time, etc.) is a core capability of online advertising and search recommendation systems. In the oCPX bidding mode, the predicted click‑through or conversion rates directly affect bidding decisions and revenue settlement, so the predictions must reflect true probabilities rather than only ranking quality.

Existing calibration approaches fall into three categories: binning‑based methods (e.g., Isotonic Regression), scaling‑based methods (e.g., Platt Scaling, Gamma Calibration), and hybrid methods (e.g., Smoothed IsoReg, NeuCalib). Binning methods preserve ordering but have limited expressive power; scaling methods assume a parametric distribution; hybrid methods combine both but still use a global calibration function.

To address these limitations, the authors propose AdaCalib, a posterior‑guided feature‑adaptive calibration model. AdaCalib incorporates posterior statistics into a deep neural network (DNN) to learn a family of calibration functions, where each distinct feature value (field value) receives its own piecewise‑linear calibration function. An adaptive bucketing mechanism ensures reliable posterior statistics for each bucket.

For a given feature domain, samples are sorted by their un‑calibrated predictions and divided into equal‑frequency buckets. Each bucket’s posterior statistics (clicks, conversions, etc.) are embedded and fed to a DNN that predicts the slope of the linear mapping for that bucket. The resulting calibration function is monotonic and piecewise linear, guaranteeing order preservation.

An auxiliary network is applied to the calibrated logits before a final logistic scaling, enhancing model expressiveness. The training objective combines cross‑entropy between calibrated outputs and true labels with a non‑negative slope constraint to enforce monotonicity.

Because feature values have highly imbalanced frequencies, AdaCalib introduces a hard‑attention (gumbel‑softmax) based adaptive bucket selector. For each feature value, a set of candidate bucket numbers is evaluated, and the selector chooses the most appropriate bucket count based on frequency‑derived embeddings, allowing low‑frequency features to use fewer buckets and high‑frequency features to use more.

During online serving, AdaCalib is merged into the original prediction model checkpoint, eliminating the need for a separate calibration module and enabling seamless inference.

Extensive offline experiments were conducted on public datasets (Ali‑CCP) and production logs (CTR and CVR) covering billions of impressions and clicks. AdaCalib was compared against IsoReg, GammaCalib, SIR, and NeuCalib using Field‑RCE, Field‑AUC, LogLoss, and AUC. Across all settings, AdaCalib achieved superior calibration accuracy (lower Field‑RCE) while maintaining or improving ranking performance (higher Field‑AUC and AUC).

Ablation studies demonstrated the importance of each component: posterior statistics, per‑feature piecewise functions, adaptive bucket selection, and the auxiliary network. Removing any of these led to noticeable performance drops.

Qualitative analyses showed that AdaCalib’s learned calibration functions align more closely with true posterior probabilities than NeuCalib, and the adaptive bucket mechanism correctly allocates more buckets to high‑frequency feature values.

In a one‑week online experiment on the RTA CVR module, AdaCalib yielded a +5.05% lift in CVR and a +5.47% increase in transaction value compared with the SIR baseline.

The paper concludes that AdaCalib is an effective first step toward integrated prediction‑calibration models for advertising, with future work focusing on adaptive feature‑domain selection and broader deployment.