Differential Privacy: Principles, Algorithms, and Applications in Data Security

The presentation begins by highlighting the challenges of protecting privacy in the era of big data, emphasizing that merely anonymizing datasets is insufficient because adversaries can reconstruct sensitive information through linkage attacks.

It then introduces differential privacy as a mathematically rigorous solution that protects both data and model privacy by ensuring that the output distribution of an algorithm remains nearly unchanged when any single individual's data is added or removed.

The formal definition states that an algorithm A satisfies ε‑differential privacy if for any two adjacent datasets D and D' (differing in one record) and for any possible output S, the probability ratio Pr[A(D)∈S] / Pr[A(D')∈S] ≤ e^ε. Smaller ε yields stronger privacy.

Three primary noise‑injection techniques are described:

Output perturbation: adding calibrated noise directly to the learned model parameters θ.

Objective perturbation: injecting noise into the loss function before optimization.

Gradient perturbation: adding noise to the gradient at each training iteration, which often preserves model utility better and can improve convergence.

A novel heterogeneous‑data differential privacy algorithm is proposed, which selectively adds noise only to data points that have a significant impact on the model, thereby reducing overall noise and improving accuracy, especially for non‑convex problems.

The article further explores the application of differential privacy in federated learning, where model parameters are exchanged instead of raw data. It discusses the non‑IID nature of client data, the challenges it poses, and how differential privacy can mitigate privacy risks while keeping communication overhead manageable.

Mathematical formulations for client data sizes n_k, distribution ρ_k, and global model aggregation are provided, along with an excess risk bound analysis that guides the redesign of loss functions for non‑IID federated settings.

Experimental results compare the proposed heterogeneous‑data DP algorithm and a federated averaging variant (FedAvgR) against classic DP methods, showing superior accuracy on both convex and non‑convex tasks, as well as reduced divergence between local and global feature distributions over training rounds.

Finally, the outlook identifies open problems such as handling non‑convex deep models and addressing non‑IID data distributions in federated learning, suggesting that further research is needed to advance privacy‑preserving machine learning.