Representation Harms › Summary

Representation harms are a distinct class of AI failure mode that operates at the level of language patterns rather than individual decisions:

• Stereotyping associates demographic groups with fixed traits, roles, or behaviors that overgeneralize from some members to all, or encode historical power structures. Template probes and WEAT-style embedding tests are the primary measurement tools. Stereotyping is not just inaccurate representation: it amplifies existing inequalities and resists correction by contextual evidence.

• Erasure is the systematic absence of groups from generated content. Corpus-level frequency analysis, comparing model outputs to population or ground truth baselines, provides the most direct measurement. The feedback loop between erasure in model outputs and erasure in future training data makes this harm self-perpetuating without active intervention.

• Demeaning associations range from explicit toxic content to subtle patterns of condescension, incompetence attribution, and hypersexualization. Toxicity classifiers, human evaluation, and counterfactual comparisons are all necessary because no single tool captures the full range. The bias in toxicity classifiers themselves against AAVE is a meta-level representation harm that suppresses Black expression under the guise of safety.

• Measurement challenges include the template coverage problem (you can only measure what you probe for), the contextualization gap between static embeddings and modern language models, and the distinction between benchmark improvement and real-world harm reduction.

• Intersectionality matters: representation harms compound across demographic dimensions and can't be fully captured by single-axis analyses. The most severe erasure typically affects groups with multiple historically marginalized identities.

• Mitigation requires interventions at multiple levels: data curation to address imbalances in training corpora, embedding-level debiasing to remove encoded associations, and generation-level techniques including prompt augmentation, constrained generation, and targeted finetuning. No single intervention is sufficient, and all interventions require ongoing monitoring to detect regression.

Representation harms illustrate why safety and fairness in language models can't be reduced to output filtering alone. The harms emerge from systematic patterns in what models generate across many interactions, making evaluation, ongoing monitoring, and community input essential components of responsible deployment. Building equitable language models requires better techniques and better practices: involving affected communities in harm definition, measurement design, and mitigation evaluation throughout the development lifecycle.