Bias Mitigation: Debiasing, CDA, and Fair Fine-tuning › Counterfactual Data Augmentation: A Worked Example
Language AI HandbookAll chapters Bias Mitigation: Debiasing, CDA, and Fair Fine-tuning › Counterfactual Data Augmentation: A Worked ExampleOne section · Sections in this chapter8 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let us implement CDA for a sentiment analysis task, showing how augmentation prevents gendered sentiment disparities.
# Swap dictionary: male -> female and female -> male
gender_swap = {
"he": "she",
"she": "he",
"him": "her",
"her": "him",
"his": "hers",
"hers": "his",
"himself": "herself",
"herself": "himself",
"man": "woman",
"woman": "man",
"men": "women",
"women": "men",
"boy": "girl",
"girl": "boy",
"boys": "girls",
"girls": "boys",
"male": "female",
"female": "male",
"father": "mother",
"mother": "father",
"son": "daughter",
"daughter": "son",
"brother": "sister",
"sister": "brother",
"husband": "wife",
"wife": "husband",
"mr": "ms",
"ms": "mr",
"mr.": "ms.",
"ms.": "mr.",
}
def swap_gender(text: str, swap_dict: dict) -> str:
"""Apply gender swap to a text string, preserving capitalization."""
tokens = text.split()
swapped = []
for token in tokens:
# Strip punctuation for lookup, preserve it
word = token.rstrip(".,!?;:").lower()
punct = token[len(word) :]
if word in swap_dict:
replacement = swap_dict[word]
# Preserve original capitalization
if token[0].isupper():
replacement = replacement.capitalize()
swapped.append(replacement + punct)
else:
swapped.append(token)
return " ".join(swapped)
# Sample training data with gendered labels
# Intentionally male-skewed (6:2) to demonstrate CDA's rebalancing effect
training_data = [
(
"He was an excellent engineer, solving every problem with confidence.",
"positive",
),
("The man quickly became the team lead after just one year.", "positive"),
("He had a commanding presence in every meeting.", "positive"),
("The boy showed remarkable aptitude for mathematics.", "positive"),
("His contributions to the project were consistently praised.", "positive"),
("The father of three managed a team of twenty engineers.", "positive"),
(
"She struggled to communicate clearly with the technical team.",
"negative",
),
("She was too emotional during the performance review.", "negative"),
]def apply_cda(data: list, swap_dict: dict) -> list:
"""Apply counterfactual data augmentation to a labeled dataset."""
augmented = list(data) # Start with original data
for text, label in data:
swapped = swap_gender(text, swap_dict)
# Only add if the swap actually changed something
if swapped != text:
augmented.append((swapped, label))
return augmented
augmented_data = apply_cda(training_data, gender_swap)Original dataset size: 8 Augmented dataset size: 16 Sample augmented pairs: ---------------------------------------------------------------------- Original: [positive] He was an excellent engineer, solving every problem with confidence. Augmented: [positive] She was an excellent engineer, solving every problem with confidence. Original: [positive] The man quickly became the team lead after just one year. Augmented: [positive] The woman quickly became the team lead after just one year. Original: [positive] He had a commanding presence in every meeting. Augmented: [positive] She had a commanding presence in every meeting.
# Analyze demographic balance after augmentation
def count_demographic_terms(data: list) -> dict:
"""Count occurrences of male and female terms in dataset."""
male_terms = {
"he",
"him",
"his",
"man",
"men",
"boy",
"boys",
"himself",
"male",
"father",
"son",
"brother",
"husband",
"mr",
}
female_terms = {
"she",
"her",
"hers",
"woman",
"women",
"girl",
"girls",
"herself",
"female",
"mother",
"daughter",
"sister",
"wife",
"ms",
}
male_count = 0
female_count = 0
for text, _ in data:
tokens = set(text.lower().split())
male_count += len(tokens & male_terms)
female_count += len(tokens & female_terms)
return {"male": male_count, "female": female_count}
counts_before = count_demographic_terms(training_data)
counts_after = count_demographic_terms(augmented_data)Demographic term balance: Dataset Male terms Female terms Balance ratio ---------------------------------------------------------- Before CDA 6 2 0.33 After CDA 8 8 1.00 A ratio of 1.0 indicates perfect gender balance.
The balance ratio approaching 1.0 after augmentation confirms that the dataset now treats male and female demographic signals with equal frequency. A model trained on this augmented dataset cannot rely on gendered pronouns as predictive features, because every example that references a man has a paired example referencing a woman with the same label.
Notice what CDA preserves as well as what it changes. The sentiment labels remain identical for original and augmented pairs, which is the correct behavior for a task where quality judgments should be gender-neutral. The vocabulary outside the swap list (words like "excellent", "commanding", "struggled", "emotional") also remains unchanged, which means the model still sees the full lexical signal in each example. CDA is a targeted operation: it corrects the demographic co-occurrence statistics without distorting any other feature of the text.
