Safety Risks › Summary

Safety risks in language models span harmful content generation, misuse by malicious actors, and unintended harms from model failures:

• Harmful content categories include violent and threatening content, CSAM, CBRN uplift, disinformation and manipulation, hate speech, privacy violations, and crisis content. Each category has different severity, different affected populations, and different mitigation strategies. Some categories warrant categorical prohibition; others require contextual calibration.
• Misuse scenarios range from low-skill, high-volume threats (automated phishing, spam, and harassment) to sophisticated adversarial attacks by actors who understand the safety system architecture. The dual-use nature of language capabilities means the same model features that enable legitimate uses also enable harmful ones, and there is no clean technical separation between them.
• Unintended harms include sycophancy and false validation, overrefusal and unhelpfulness, representational and allocational biases, and long-tail failures that surface at deployment scale. These harms arise from training dynamics and deployment gaps rather than from adversarial intent, and they require different solutions from harms caused by deliberate misuse.
• Threat models structure thinking about safety by asking who the adversary is, what capabilities they have, what they want to achieve, and how much the model increases their capability. Concepts like uplift and counterfactual impact help calibrate which safety interventions are worth their costs to legitimate users.
• The Risk = P x M x B framework provides a principled basis for asymmetric treatment: low-probability, catastrophic, broad harms warrant extraordinary precaution even when their expected occurrence is rare. Irreversibility strengthens this precautionary argument further.
• Safety is inherently a tradeoff between preventing harmful outputs and maintaining usefulness. There is no threshold that achieves both simultaneously; safety design requires choosing an operating point appropriate to the deployment context, and overrefusal is itself a harm that belongs on the same analytical footing as the harms it prevents.

The subsequent chapters in Part LX build on this foundation. Red teaming provides systematic methods for discovering where safety systems fail. Jailbreaking catalogs the techniques adversaries use to bypass safety constraints. Content filtering and guardrails offer the practical tools for implementing safety at deployment time.