Differential Privacy › Summary

Differential privacy gives machine learning engineers a principled, mathematically rigorous way to limit how much any single training example influences the final model. The core idea is to bound the sensitivity of gradients via clipping and then add calibrated Gaussian noise, giving the DP-SGD algorithm. The privacy cost accumulates across training steps and is tracked via a privacy accountant, with the total cost expressed as $(\varepsilon, \delta)$: smaller $\varepsilon$ means stronger privacy.

The key technical ingredients are:

• The $(\varepsilon, \delta)$ guarantee: bounds the log-odds ratio by which the model can reveal whether any particular record was included in training.
• Gradient clipping: bounds the sensitivity of the training algorithm by ensuring no single example can shift the gradient aggregate by more than $C$ in $\ell_2$ norm.
• Gaussian noise: adds calibrated noise scaled to $\sigma C$ to hide the contribution of any clipped gradient within the noise floor.
• Privacy accounting: tracks the cumulative privacy cost via the moments accountant or RDP accountant, enabling precise budget management.
• Subsampling amplification: the fact that each example participates in only a fraction $q$ of training steps provides free privacy amplification, making the effective per-step cost roughly $O(q \varepsilon)$ rather than $\varepsilon$.

Key takeaways for practitioners:

• DP guarantees are worst-case: Unlike heuristic defenses, DP protection holds even against adversaries with arbitrary auxiliary knowledge and computation.
• The privacy budget is additive: Every training step spends epsilon, and the total cost must be accounted for and reported, including hyperparameter tuning experiments.
• Scale is your friend: Large models and large datasets dramatically improve the privacy-utility tradeoff. DP training is most practical when fine-tuning large pre-trained models on private downstream data.
• DP-LoRA is a practical default: Combining DP with parameter-efficient methods like LoRA reduces noise dimensionality and provides better accuracy at a given epsilon.
• Fairness and hyperparameter tuning are real concerns: DP disproportionately hurts minority subgroups, and tuning experiments consume privacy budget that must be counted.
• Empirical auditing is essential: Theoretical guarantees can be invalidated by implementation bugs; canary-based auditing provides empirical validation.

The next chapter explores interpretability techniques for understanding what language models have learned, which is the other side of the accountability coin: differential privacy tells us how much individuals influenced the model, while interpretability tells us what the model knows.