GBA: Global Batch Gradients Aggregation for Search Advertising Training

This paper introduces GBA (Global Batch Gradients Aggregation), a novel training mode that enables seamless switching between synchronous and asynchronous training in search advertising models. By maintaining a consistent global batch size during gradient aggregation, GBA addresses precision loss and parameter divergence issues during mode transitions. The approach combines token-controlled gradient aggregation with staleness management to ensure stability in mixed-resource environments. Experimental results demonstrate GBA achieves synchronization-level accuracy while preserving asynchronous efficiency, with applications in large-scale sparse model training. The framework supports automatic mode switching without hyperparameter adjustments, offering significant advantages for industrial deployment.

Key contributions include convergence analysis proving GBA's theoretical equivalence to synchronous training under specific conditions, engineering implementation details for token management and gradient buffering, and empirical validation across multiple advertising models. The method enables efficient iteration in both high-performance and standard computing environments, reducing dependency on legacy parameters and accelerating model development cycles.