Representation Harms › Demeaning Associations

Demeaning associations occur when a model generates content that degrades, dehumanizes, or expresses contempt toward a group. This category spans a wide range: from explicit slurs and hate speech at one extreme to subtler patterns of condescension, infantilization, and association with negative attributes at the other.

Language models can generate explicit slurs, hate speech, and demeaning characterizations, particularly when prompted with template completions, roleplay scenarios, or requests to "speak as" particular characters. Research on models like GPT-2 and its successors showed that models could be reliably prompted to generate racist, sexist, or otherwise demeaning content through techniques that bypass explicit content filters.

This category has received the most attention from moderation and safety teams, partly because it's the most visible and partly because the harm is clearest. Explicit demeaning content can be measured directly with classifiers trained on labeled toxicity data, and its presence in model outputs is easily demonstrable. However, focusing exclusively on explicit toxic content risks creating models that are "polite bigots": models that don't produce slurs but still systematically encode demeaning patterns through the subtler mechanisms described below.

More difficult to detect, and arguably more pervasive, are subtle demeaning associations that don't rise to the level of explicit slurs but systematically portray certain groups in degrading ways:

• Competence disparagement: Models describing characters from lower-status demographic groups as confused, forgetful, or incompetent at higher rates than comparable characters from higher-status groups
• Infantilization: Describing adults from certain groups with child-like framing, using diminutives, or attributing naivety
• Hypersexualization: Generating sexually focused descriptions for women and for certain racial groups in professional contexts where such descriptions would be inappropriate
• Agency erasure: Describing group members as passive, acted upon, or dependent rather than as active agents with goals and decisions
• Credibility discounting: Generating narratives where characters from certain groups have their professional judgments questioned or overridden more frequently than equivalent characters from other groups

These patterns are harder to catch with simple classifiers because they depend on context: describing a character as "childlike" in one scenario might be neutral, in another condescending. Detecting them requires discourse-level analysis rather than token-level toxicity scoring. Researchers typically use a combination of carefully designed templates, embedding-based sentiment analysis, and human annotation to identify and measure these patterns.

The harm from subtle demeaning patterns is real even though it operates below the threshold of explicit offensiveness. When a model consistently portrays women in professional contexts as needing validation or guidance from male colleagues, even in language that is never harsh or vulgar, it reinforces the idea that female professional authority is contingent rather than inherent. The accumulation of thousands of such portrayals in the outputs of widely used tools shapes professional culture.

A consistent finding across language model studies is that models apply different sentiment and quality standards to text from different demographic groups. In toxicity detection tasks, text written in AAVE is more likely to be classified as toxic than equivalent text written in Standard American English, even when the semantic content is the same. In translation tasks, models apply more fluent and natural-sounding outputs when translating into English from certain source languages than from others.

These asymmetries are themselves demeaning: they encode the idea that some ways of using language are inherently better than others, where "better" maps onto racial and cultural hierarchies. When a model's quality scorer assigns lower scores to AAVE text, it's not measuring quality in any culture-neutral sense. It's measuring deviation from a standard that was defined by the dominant culture, and then treating that deviation as a defect.

The Sap et al. (2019) study documented this pattern directly: annotations of tweets written in AAVE for a toxicity dataset showed that AAVE tweets were annotated as toxic at roughly 3-4 times the rate of equivalent tweets written in Standard American English. Because these annotations became training data for toxicity classifiers, the bias propagated from human annotators into model outputs. Classifiers trained on this data then produce higher toxicity scores for AAVE text, which leads to systematic over-filtering of Black social media content by safety systems.

It's useful to think about demeaning associations along a spectrum from explicit to implicit, because the measurement and mitigation strategies differ:

At the explicit end, content contains direct slurs, dehumanizing language, or overt expressions of contempt. These are measurable with relatively simple classifiers and their harm is unambiguous. At the implicit end, content uses polite language but encodes demeaning patterns through framing, omission, and comparison. These are harder to measure, require discourse-level analysis, and their harm is easier to rationalize away.

The implicit end of the spectrum is where deployed models most often cause harm in practice. Explicit toxicity is aggressively filtered in most production systems. Implicit demeaning patterns slip through because they don't trigger content filters and because individual outputs seem innocuous. The harm is realized only across the pattern, not within any individual generation, which is exactly what makes it difficult to surface in standard testing.