Representation Harms › Mitigation Strategies

Knowing how to measure representation harms is only half the problem. The other half is what to do about them. Mitigation strategies differ substantially depending on which harm type is being addressed and which stage of the model pipeline the intervention targets.

The most upstream interventions target training data before the model ever sees it. Counterfactual data augmentation (CDA) systematically swaps demographic references in training texts to create balanced datasets: for every sentence containing "male nurse," a counterfactual "female nurse" sentence is added. This reduces the strength of gender-occupation associations in the learned embeddings without changing the content of the training data in any other way.

A more aggressive version of data augmentation is resampling: increasing the proportion of underrepresented groups in training data by oversampling or by targeted data collection. For historical and cultural erasure, this might mean seeking out and including texts from underrepresented historical traditions, non-Western knowledge systems, and community-produced content from marginalized groups.

The limitation of data-level interventions is that they address proximate causes rather than root causes. The training data reflects real-world inequality, and debiasing the data doesn't change that underlying inequality. A model trained on augmented data that perfectly balances gender in occupational roles may produce more equitable outputs, but it's not an accurate model of the current world. Researchers and practitioners need to be explicit about whether they're optimizing for equitable representation or accurate representation, and to recognize that these goals can conflict.

Bias in the representation space can be directly targeted through debiasing algorithms applied to trained embeddings. The Hard Debiasing algorithm of Bolukbasi et al. (2016) identifies the gender subspace in word embeddings using a set of definitionally gendered word pairs (man/woman, king/queen, etc.), then projects out the gender component from neutral words like occupational terms.

More formally, if $\mathbf{g}$ is the gender direction in the embedding space (typically the first principal component of the differences between gender-paired word vectors), the projection of a neutral word vector $\mathbf{w}$ onto the gender direction is:

$$\mathbf{w}_{\text{gender}} = \frac{\mathbf{w} \cdot \mathbf{g}}{\|\mathbf{g}\|^2} \mathbf{g}$$

and the gender-debiased word vector is:

$$\mathbf{w}_{\text{debiased}} = \mathbf{w} - \mathbf{w}_{\text{gender}}$$

This operation removes the gender component from neutral words while preserving their other semantic properties. After debiasing, "programmer" and "nurse" should have near-zero projections onto the gender direction, while "man" and "woman" remain fully gendered.

The limitation of hard debiasing is that it targets a defined bias direction, which requires specifying in advance what counts as gender and what counts as neutral. The gender subspace identified from a handful of word pairs may not capture all the ways gender is encoded in the embedding space. More fundamentally, some studies have found that hard debiasing reduces WEAT scores without reducing stereotype-based behavior in downstream tasks: the bias is hidden from the measurement rather than removed from the model.

For large language models where modifying the embedding space directly is impractical, generation-level interventions can reduce representation harms at inference time. These include:

Prompt augmentation adds demographic diversity instructions to system prompts, e.g., "When describing characters, vary their demographic backgrounds." This is simple but unreliable: models may follow demographic diversity instructions for some prompts and ignore them for others.

Constrained generation uses techniques like classifiers or rerankers to filter or rerank model outputs to exclude or de-emphasize outputs that score poorly on representation metrics. This is more reliable than prompt augmentation but adds latency.

Targeted finetuning trains the model on curated demonstrations that exhibit equitable representation, either through supervised examples or through reinforcement learning from human feedback (RLHF) with raters trained to identify and penalize representation harms. This is the most effective approach but also the most resource-intensive, and its effects can decay or generalize poorly to new domains.