Jailbreaking › The Alignment Tax and Robustness Tax

An important practical consideration is that safety mechanisms have costs beyond the obvious computational ones. Two concepts capture these costs.

The alignment tax refers to capability degradation from safety training. A model that refuses more harmful requests may also refuse more borderline legitimate requests, reducing its usefulness. Early RLHF implementations showed that heavily aligned models were noticeably more cautious and sometimes less capable at complex tasks than their base model counterparts. The model trained to say "I can't help with that" for harmful requests sometimes said it for ambiguous legitimate requests too, because the learned refusal behavior generalized too broadly.

Research has worked to minimize the alignment tax while maintaining safety. Modern aligned models are considerably more capable than early aligned versions, suggesting that the tax is not fundamental but rather a training efficiency issue. Larger models, more carefully curated training data, and better reward model design have all contributed to reducing over-refusal rates. However, trade-offs remain, especially in specialized domains where legitimate content closely resembles potentially harmful content: medical information, cybersecurity research, weapons policy analysis, and historical documentation of atrocities all require models that can engage with difficult material without refusing reflexively.

The robustness tax is the performance hit from adding defenses. Perplexity filtering rejects some legitimate queries. Output filtering adds latency. Randomized smoothing requires many inference passes. Each defensive layer degrades either response quality or throughput.

System designers must decide how much of each tax they can accept given their threat model. A model deployed for medical information retrieval has a different optimal operating point than a model used for creative writing. The former tolerates more false positives to ensure no dangerous medical advice escapes; the latter tolerates fewer false positives to maintain creative freedom. This suggests that safety posture should be customized per deployment rather than applied uniformly, which is exactly the design philosophy behind system prompts and fine-tuning for specific use cases.