Vertical Federated XGBoost (XGB) Algorithm: Problem Definition, Secure Training, Optimization, and Prediction

Speaker Fang Wenjing, an algorithm expert from Ant Group, introduces the topic "Vertical Federated XGB Algorithm" focusing on privacy‑preserving joint modeling across multiple computing nodes.

Problem Definition: The growing demand for privacy computing highlights the need for vertical federated learning of XGB tree models, yet academic research on vertical XGB is scarce and industrial deployments often expose intermediate information such as sample distributions and gradient sums.

In XGBoost, the tree model consists of two node types: intermediate nodes storing split features and thresholds, and leaf nodes storing weight values. The split‑gain for each feature is computed using the formula shown below.

Algorithm Setting: Secure XGB modifies the input and output formats. Input data are vertically partitioned and aligned across parties. Output uses a distributed XGB model where split information is kept by the owning party and leaf weights are secret‑shared among participants.

Secure Training: The training leverages traditional Secure Multi‑Party Computation (MPC) techniques, primarily secret sharing and homomorphic encryption (HE). Secret sharing distributes random shares of data to parties, while HE enables encrypted computation with low communication overhead.

The proposed SS‑XGB algorithm converts variables to secret‑shared form and improves two core operations:

Bucket Accumulation: (1) Generate a flag vector indicating which samples fall into a bucket; (2) Secret‑share the flag vector; (3) Compute the inner product with the gradient vector, ensuring the other party cannot reconstruct the flag values.

Sample Partition: Protect sample distribution by zeroing gradients of samples not belonging to a split subtree.

Bottleneck Analysis: The flag vector is sparse, leading to high computational cost. An oblivious permutation operator is introduced to mitigate this overhead.

The oblivious permutation sorts secret‑shared gradients using a permutation function (shown below), enabling two optimized variants:

HEP‑XGB: Utilizes HE to perform bucket accumulation. Two conversion protocols are defined: (1) H2S – transforms HE ciphertexts into secret‑shared values by subtracting a random share; (2) S2H – encrypts secret‑shared values back to ciphertexts for further computation.

CRP‑XGB: Introduces random masks generated in a trusted execution environment (TEE) to mask secret‑shared vectors, allowing the sorting problem to be solved in cleartext while keeping the original secret shares protected, thus improving efficiency especially under good network conditions.

Prediction Algorithm: In a single‑tree plaintext prediction, a one‑hot flag vector is multiplied by global weights. In vertical federated XGB, each party generates a local prediction vector, converts it to secret‑shared form, performs element‑wise multiplication across parties, and finally computes the inner product with leaf weights to obtain the prediction.

Performance tests under three network scenarios (S1: 32‑core, 10 Gbps, 0.1 ms; S2: 32‑core, 100 Mbps, 5 ms; S3: 8‑core, 20 Mbps, 50 ms) show that CRP‑XGB is ~10× faster than HEP‑XGB in the best network, while HEP‑XGB saves ~26.5 % time in poorer networks. Large‑scale experiments with 12 million samples and 200‑dimensional features demonstrate that both algorithms scale to millions of samples, with CRP‑XGB consuming less CPU and memory and achieving shorter training times when the network is favorable.

In conclusion, the vertical federated XGB framework, together with the SS‑XGB, HEP‑XGB, and CRP‑XGB variants, provides a practical solution for privacy‑preserving large‑scale tree model training and inference.

Thank you for attending.