Bias Mitigation: Debiasing, CDA, and Fair Fine-tuning › Evaluating Mitigation Effectiveness
Language AI HandbookAll chapters Bias Mitigation: Debiasing, CDA, and Fair Fine-tuning › Evaluating Mitigation EffectivenessOne section · Sections in this chapter9 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Every mitigation technique requires validation. It is entirely possible to apply a debiasing method that reduces one form of bias while inadvertently increasing another, or that reduces bias on the test set while degrading task performance unacceptably. Evaluation closes the loop.
We will explore fairness metrics in depth in the next chapter, but three measurements are essential for any mitigation evaluation.
Task performance retention. Report the main task metric (accuracy, F1, BLEU, etc.) on a held-out evaluation set before and after mitigation. An acceptable mitigation degrades task performance by no more than a few percentage points. If performance drops by 10% or more, the mitigation is too aggressive.
Embedding association tests. For embedding debiasing, rerun the Word Embedding Association Test (WEAT) scores from Bias Measurement after applying the fix. The effect size should decrease significantly for the targeted associations.
Counterfactual consistency. For fine-tuning and prompt-based methods, measure how much model outputs change when you swap demographic attributes in test inputs. A perfectly mitigated model would show zero change; in practice, you aim for a substantial reduction in variability.
def counterfactual_consistency_score(
texts: list, predict_fn, swap_fn, num_samples: int = None
) -> dict:
"""
Measure how consistently a model treats demographic variants.
Returns the average absolute difference in predicted probabilities
between original and counterfactually swapped inputs.
A score of 0.0 indicates perfect consistency.
"""
if num_samples is not None:
texts = texts[:num_samples]
differences = []
swapped_count = 0
for text in texts:
swapped = swap_fn(text)
if swapped == text:
continue # No demographic terms to swap
prob_original = predict_fn(text)
prob_swapped = predict_fn(swapped)
diff = abs(prob_original - prob_swapped)
differences.append(diff)
swapped_count += 1
if not differences:
return {"score": 0.0, "n_evaluated": 0}
return {
"mean_difference": float(np.mean(differences)),
"max_difference": float(np.max(differences)),
"std_difference": float(np.std(differences)),
"n_evaluated": swapped_count,
}
# Simulate a biased model and a debiased model for demonstration
def biased_model(text: str) -> float:
"""Simulate a model with gender bias: positive sentiment for male-coded text."""
male_terms = {"he", "him", "his", "man", "men"}
female_terms = {"she", "her", "hers", "woman", "women"}
tokens = set(text.lower().split())
score = 0.5 # Base probability
if tokens & male_terms:
score += 0.2
if tokens & female_terms:
score -= 0.1
return min(1.0, max(0.0, score))
def debiased_model(text: str) -> float:
"""Simulate a debiased model: gender-neutral predictions."""
return 0.55 # Same prediction regardless of gendered terms
test_texts = [text for text, _ in training_data]
swap_fn = lambda t: swap_gender(t, gender_swap)
biased_scores = counterfactual_consistency_score(
test_texts, biased_model, swap_fn
)
debiased_scores = counterfactual_consistency_score(
test_texts, debiased_model, swap_fn
)Counterfactual Consistency Evaluation Metric Biased Model Debiased Model ---------------------------------------------------------- Mean |difference| 0.225 0.000 Max |difference| 0.300 0.000 Std |difference| 0.130 0.000 Samples evaluated 8 8
The mean difference represents the average change in predicted probability when demographic attributes are swapped. Lower is better. We will formalize this into demographic parity and equalized odds metrics in the Fairness Metrics chapter.
One important note about this evaluation framework: counterfactual consistency measures a necessary but not sufficient condition for fairness. A model that predicts 0.5 probability for every input is perfectly counterfactually consistent (the score never changes when you swap demographic attributes), but it has also learned nothing useful. Always pair counterfactual consistency evaluation with task performance metrics to ensure the model has not achieved demographic invariance by collapsing to a constant predictor. The two metrics together tell the full story: consistency checks that the model does not use demographic signals, while accuracy checks that it still makes meaningful predictions.
Key Parameters
The key parameters for the techniques implemented in this chapter are:
- swap dictionary: The set of demographic term substitutions applied by CDA. A larger dictionary catches more demographic signals but may introduce errors in ambiguous contexts (e.g., "her" as a possessive vs. a pronoun).
- PCA n_components: The number of principal components used to model the bias subspace. Setting this to 1 captures the dominant gender axis; setting it to 2-3 may capture additional dimensions (e.g., formality, age) that correlate with gender in the training corpus.
- lambda (): The fairness regularization strength in soft debiasing and the adversarial loss weighting in adversarial training. Higher values enforce stronger debiasing but risk degrading task performance. Typical values range from 0.1 to 1.0.
- gendered seed pairs: The definitionally gendered word pairs used to estimate the bias direction. Quality matters more than quantity: 10-20 precise pairs outperform 50 noisy pairs for accurate bias direction estimation.