Bias Mitigation: Debiasing, CDA, and Fair Fine-tuning › The Anatomy of a Mitigation Strategy

Before diving into specific techniques, it helps to think clearly about what we are trying to achieve. Mitigation is not a single operation but a pipeline decision. To select the right technique, you need to answer three questions.

Where does the bias manifest? Bias in static word embeddings (Word2Vec, GloVe) calls for different fixes than bias in a generative language model or a classification head. The source determines the remedy. A sentiment classifier that produces disparate scores across demographic groups may have a biased feature extractor, a biased training set, or a biased label distribution, and each root cause demands a different intervention. Debiasing the embeddings will not help if the training labels themselves encode discriminatory decisions.

What kind of bias are you targeting? As we discussed in Bias in Language Models, bias takes many forms: stereotypical associations, representation gaps, sentiment disparities, and toxicity amplification. Some techniques address one form; others are general. CDA, for instance, directly targets any prediction that depends on demographic identity words, while corpus filtering might specifically target representational gaps without affecting stereotypical framing in the surviving data.

What are your constraints? If you can retrain from scratch with a curated dataset, you have maximum flexibility. If you are working with a closed API, prompt-based techniques are your only option. If you have compute but not data, adversarial fine-tuning may be your best bet. For teams deploying production models, the constraint question is often decisive: a team with a 70B-parameter model accessed via an external API can only do prompt engineering, regardless of how much they wish they could retrain the embeddings.

Understanding these constraints up front prevents a common failure mode: investing engineering effort in a powerful mitigation technique that turns out to be inapplicable given the actual deployment context. With those questions answered, the techniques in this chapter will make more sense as tools selected deliberately rather than magic spells applied hopefully.

A fourth question is worth adding: what is your validation plan? Every mitigation technique introduced in this chapter can make things worse in subtle ways if applied carelessly. Hard debiasing can reduce cosine similarity between semantically related words, harming downstream retrieval tasks. Adversarial training can converge to a degenerate solution where the model collapses all representations to a constant. CDA can inadvertently label-flip examples if the underlying annotation process was demographically inconsistent. None of these failure modes are hypothetical; they have appeared in production systems. Knowing before you start how you will measure both the targeted improvement and potential regressions keeps you from declaring victory prematurely.