Safety Risks › Visualizing Safety Risk Categories

The tradeoff curve makes the safety design tension concrete. As the refusal threshold decreases (moving the system toward greater caution), harmful output rates drop but false positive rates climb. There is no threshold that simultaneously achieves zero harmful outputs and zero false positives because the distributions of harmful and benign requests overlap. Any safety system must choose an operating point based on the relative costs of these two error types in its specific deployment context.

The Risk = P x M x B framework becomes intuitive when plotted. Each harm category can be positioned in a two-dimensional space of probability (how often it occurs) and magnitude (how severe it is when it does). Categories in the upper right are both common and severe; categories in the upper left are rare but potentially catastrophic. The area of each marker reflects the breadth dimension: how many people are affected.

The chart reveals the asymmetry that drives safety policy. CBRN uplift and CSAM sit in the high-magnitude corner despite low probability, while disinformation and fraud cluster in the high-probability, moderate-magnitude zone. The iso-risk contours (dashed lines) show that points far into the low-probability, high-magnitude region can have the same expected risk product as more common, moderate harms. This is why safety policies treat rare but catastrophic categories with extraordinary precaution: the iso-risk logic justifies it even under expected value calculations, and irreversibility strengthens the case further.

Notice also that overrefusal appears in the chart as a harm category in its own right. This is deliberate. Treating excessive caution as a form of harm places it on the same analytical footing as the harms it is designed to prevent. A safety system that succeeds at minimizing harmful outputs while generating a high overrefusal rate has not achieved safety; it has traded one type of failure for another.