UKD: Debiasing Conversion Rate Estimation via Uncertainty-regularized Knowledge Distillation

This paper presents a method called UKD (Uncertainty-Regularized Knowledge Distillation) to address sample selection bias (SSB) in post-click conversion rate (CVR) estimation for online advertising systems. The approach leverages both clicked and unclicked samples by generating pseudo-conversion labels for unclicked samples through a click-adaptive teacher model. The student model then trains on the full space using these pseudo-labels while incorporating uncertainty constraints to mitigate noise in the pseudo-labels.

The teacher model uses adversarial learning to learn click-invariant feature representations, generating pseudo-labels for unclicked samples. The student model combines clicked samples (with true labels) and unclicked samples (with pseudo-labels) during training. Uncertainty estimation is applied to pseudo-labels to dynamically adjust their influence on the student model's training, reducing the impact of noisy labels.

Experiments on large-scale production datasets (ranging from 200M to 800M samples) and a public dataset (Ali-CCP) show UKD achieves significant improvements in CVR, CTCVR, and CPA metrics compared to state-of-the-art methods. Online experiments in real-world scenarios also demonstrate practical effectiveness, with CVR improvements up to 3.4% and CPA reductions of 4.3%.

The method's key innovations include the click-adaptive teacher model for pseudo-label generation, uncertainty-regularized distillation to handle label noise, and end-to-end training of the student model on the full space. These components collectively address the SSB problem by enabling effective utilization of unclicked samples in CVR estimation.