Bias Mitigation: Debiasing, CDA, and Fair Fine-tuning › Data Balancing

The most direct cause of bias is imbalanced training data. If your corpus contains ten times as many examples of men in leadership roles as women in the same roles, the model will learn that association as a statistical regularity rather than a social artifact. Data balancing attacks the problem at the source.

The simplest approach is to audit your training corpus and remove or reweight documents that exhibit strong demographic skew. This requires defining what "skew" means in your context.

For a text corpus, common filtering criteria include:

• Demographic imbalance: sentences where a profession or role is associated almost exclusively with one gender or ethnicity
• Stereotypical framing: documents where minority groups appear primarily in negative or limited contexts
• Representation gaps: entire domains (scientific literature, technical writing) that historically over-represent certain demographics

Filtering has a clean implementation story but a significant limitation: you cannot remove bias from text without removing information. A corpus filtered of all gender-profession associations will also be filtered of legitimate examples like "the first female astronaut" or "historically, nursing was considered women's work." Context matters, and simple heuristics will censor both the bias and the history.

Practical corpus curation involves at least two stages. In the first stage, you flag candidate documents using automated tools, such as measuring the ratio of male-coded to female-coded pronoun usage per document, or computing the co-occurrence strength between profession words and demographic markers. In the second stage, human reviewers examine the flagged documents to determine whether the demographic signal reflects a structural problem (e.g., a dataset scraped predominantly from male-authored sources) or informative historical content that should be preserved. Automating the first stage is tractable; fully automating the second is not, because it requires contextual judgment that keyword statistics cannot provide.

One practical heuristic: bias-aware curators often prefer reweighting documents over removing them. If a particular news source publishes articles that consistently associate women with domestic roles rather than professional ones, you can downweight that source relative to sources with more balanced coverage. The model still trains on the content, but its statistical weight in the learned distribution is reduced. This is gentler than outright removal and reduces the risk of inadvertently deleting valuable information.

Reweighting also has a diagnostic advantage over deletion. Because the original documents remain in the pipeline (just with reduced weight), you can compare the model trained with reweighted data against a baseline more cleanly than you can when data has been removed. If the reweighted model performs unexpectedly, you can increase the downweighted source's contribution and observe the effect. Deletion makes those experiments harder to reverse.

Counterfactual Data Augmentation (CDA) takes a more surgical approach. Rather than removing biased examples, CDA generates counterpart examples that swap demographic identifiers. For every sentence mentioning a man in a role, you add a synthetic sentence mentioning a woman in the same role.

The intuition is elegant: if the model sees "The doctor examined his patient" and "The doctor examined her patient" with equal frequency, it cannot learn a directional association between the doctor role and maleness. The gendered pronoun becomes statistically independent of the profession.

The formal setup is straightforward. Given a dataset $D = \{(x_1, y_1), \ldots, (x_n, y_n)\}$ where each $x_i$ contains demographic terms, CDA applies a swap function $\phi$ to produce augmented pairs:

$$D_{\text{aug}} = D \cup \{(\phi(x_i), y_i) : x_i \in D\}$$

where $\phi$ maps each demographic term to its counterpart according to a swap dictionary (e.g., "he" $\to$ "she", "John" $\to$ "Jennifer").

The swap dictionary is the critical engineering artifact. A minimal dictionary for gender might look like:

For more sophisticated augmentation, name lists are particularly valuable. Names carry strong demographic signals in text: "Jamal applied for the loan" and "Greg applied for the loan" will generate different predictions from a biased model even when all other features are identical. Adding name-swapping augmentation directly addresses this. Research by Bertrand and Mullainathan (2004) in the labor economics literature established that resume callbacks differ significantly by name ethnicity even with identical qualifications, and subsequent NLP work has demonstrated that language models reproduce these same patterns. Name-based CDA provides a direct counterfactual for testing and correcting this specific form of bias.

CDA has an important assumption baked into its design: that swapping demographic terms preserves the ground-truth label. This assumption holds well for sentiment analysis ("she is an excellent manager" and "he is an excellent manager" should both be positive), but it can break down in other tasks. If your labels were assigned by human annotators who held implicit biases, the "correct" label may already reflect those biases. In that case, CDA generates counterpart examples with the same biased label, and the model learns demographic-neutral bias. Inspecting the label distribution by demographic subgroup before applying CDA is therefore a prerequisite, not an optional step.

Another practical consideration is the scope of the swap dictionary. A dictionary that covers pronouns and common names will address the most salient demographic signals but leave subtler markers intact. Word choice, sentence structure, and topic selection can all correlate with demographic attributes without containing a single item from the swap list. CDA is a necessary but not sufficient component of a broad mitigation strategy.

CDA also carries a distinct risk with languages that have grammatical gender or complex agreement systems. In English, swapping "he" to "she" is typically safe because English pronouns do not force agreement changes elsewhere in the sentence. In Spanish, French, or German, demographic identity affects multiple word forms throughout the sentence. A naive swap dictionary in these languages produces ungrammatical augmented examples, which teaches the model to associate grammatical errors with one demographic group. Applying CDA to multilingual or non-English data requires language-specific tools that handle agreement transformations, not just lexical substitution.

When full corpus rebalancing is impractical, re-sampling adjusts the effective weight of examples during training. Two strategies cover most use cases.

Upsampling increases the frequency of underrepresented examples by sampling them with replacement. If minority demographic groups appear in 15% of your labeled data, you can upsample those examples until they represent 50%, training the model to weight them equally. Upsampling is conceptually simple and straightforward to implement with any training framework, but it does introduce repeated gradients for the upsampled examples. If the underrepresented examples share systematic characteristics, repeated exposure can cause the model to overfit those particular patterns rather than generalizing across the broader demographic distribution.

Importance weighting assigns each example a scalar weight $w_i$ that modulates its contribution to the loss:

$$\mathcal{L}_{\text{weighted}} = \frac{1}{n} \sum_{i=1}^{n} w_i \cdot \ell(f(x_i), y_i)$$

where:

• $n$: total number of training examples
• $w_i$: scalar weight for example $i$, typically set inversely proportional to the frequency of the demographic group represented in $x_i$ (rarer groups receive higher weights)
• $\ell(f(x_i), y_i)$: the per-example loss, such as cross-entropy between the model's prediction $f(x_i)$ and the true label $y_i$

This allows you to rebalance the effective training distribution without physically duplicating data, which is especially valuable for large datasets where duplication would be computationally prohibitive.

A practical advantage of importance weighting over upsampling is that each training example appears exactly once per epoch, so the gradient update variance remains controlled. The weight $w_i$ is computed once during data loading and used consistently throughout training. The most common weighting scheme sets $w_i$ inversely proportional to the class frequency of the demographic group: if group $g$ has frequency $p_g$ in the training set, every example from group $g$ receives weight $w_i = 1 / p_g$. This inverse-frequency weighting exactly corrects the distribution mismatch between the training set and a target distribution where all groups are equally represented.

Both upsampling and importance weighting are blind to the quality of the underrepresented examples. If the underrepresented group's examples are noisy, mislabeled, or systematically different in ways beyond the demographic attribute, resampling will amplify those differences rather than correct them. Always audit the underrepresented examples before applying any resampling strategy.