Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Counterfactual Fairness

Michael BrenndoerferMarch 16, 20264 min read

Language AI HandbookAll chapters

Part 58 of 67Bias and Fairness

Chapter 444Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Counterfactual Fairness

Reading section 6 of 13 · Counterfactual Fairness · 4 minSection 6 of 13 · 4 min

Section 6 of 13 · 4 min read

Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Counterfactual FairnessOne section · Sections in this chapter6 / 13 ↓

Sections in this chapter13

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Counterfactual fairness asks: would my decision change if this person belonged to a different demographic group, holding everything else constant?

Counterfactual Fairness

A predictor $\hat{Y}$ is counterfactually fair if, for any individual with observed features $X = x$ and sensitive attribute $A = a$ :

P(\hat{Y}_{A \leftarrow a}(U) = y \mid X = x, A = a) = P(\hat{Y}_{A \leftarrow a'}(U) = y \mid X = x, A = a)

where $\hat{Y}_{A \leftarrow a'}$ denotes the counterfactual prediction had the individual been in group $a'$ , and $U$ represents unmeasured background variables.

This definition, from Kusner et al. (2017), uses causal reasoning. The notation $A \leftarrow a'$ means "imagine a world where person $i$ 's sensitive attribute is changed to $a'$ , but everything causally unrelated to $A$ remains the same." If the prediction does not change under this counterfactual intervention, the classifier is counterfactually fair.

The difficulty is that computing counterfactuals requires a causal model of how the features are generated. If race affects which schools are available, which GPA scale is used, and which neighborhoods a person lives in, then a true counterfactual requires specifying all of these downstream effects. Changing only the sensitive attribute while holding everything else fixed is only coherent if you have a complete causal model that separates what changes from what stays the same when you change $A$ .

Naive approaches that simply remove the sensitive attribute from the feature set fail because other features may encode the same information through causal pathways. In our hiring example, removing the "group" column does not prevent the model from learning to use reported skill scores as a proxy, because reported skill scores were generated with a group-specific offset. Removing a proxy leaves the information in the data; it just becomes harder to trace. This is the proxy problem that causal fairness approaches are designed to address.

Counterfactual fairness is appealing because it aligns with legal intuitions: a person's protected status should not determine their outcome, and the way to check this is to ask what would have happened if they belonged to a different group. But it is computationally demanding and requires causal assumptions that are often difficult to validate from observational data alone. In practice, researchers use structural causal models (SCMs) to operationalize counterfactual fairness, identifying which features are direct descendants of $A$ in the causal graph and which are not. Features that are not causally influenced by $A$ can be used safely; features that flow through causal pathways from $A$ require special treatment.

Let's demonstrate a simplified counterfactual fairness check. We will test whether flipping the group indicator (while holding the other features constant at their observed values) changes the model's prediction.

In[18]:

Code

def counterfactual_fairness_check(
    clf, X_test, group_test, feature_col_is_group=None
):
    """
    Approximate counterfactual fairness by testing whether predictions change
    when the group-correlated feature (reported_skill, column 0) is adjusted
    to remove the group-specific offset.

    This simulates asking: if a Group 1 individual had the same reported_skill
    score as they would have received without the group penalty, would the
    prediction change?
    """
    group_penalty = 0.5  # The offset used during data generation

    X_cf = X_test.copy()
    # For Group 1, add back the artificially subtracted penalty
    group1_mask = group_test == 1
    X_cf[group1_mask, 0] += group_penalty

    y_pred_original = clf.predict(X_test)
    y_pred_counterfactual = clf.predict(X_cf)
    y_prob_original = clf.predict_proba(X_test)[:, 1]
    y_prob_counterfactual = clf.predict_proba(X_cf)[:, 1]

    # Compare for Group 1 only
    changed_decisions = (
        y_pred_original[group1_mask] != y_pred_counterfactual[group1_mask]
    )
    prob_shift = (
        y_prob_counterfactual[group1_mask] - y_prob_original[group1_mask]
    )

    return {
        "fraction_changed": changed_decisions.mean(),
        "mean_prob_shift": prob_shift.mean(),
        "n_group1": group1_mask.sum(),
        "n_changed": changed_decisions.sum(),
        "prob_shift": prob_shift,
    }


cf_results = counterfactual_fairness_check(clf, X_test, group_test)

Out[19]:

Console

=== Counterfactual Fairness Check (Group 1) ===
Group 1 individuals tested: 401
Decisions that would change: 52 (13.0%)
Mean probability shift when bias removed: +0.079

Interpretation: 13.0% of Group 1 individuals
would receive different decisions if the group-specific
bias in reported_skill were removed.

The counterfactual check quantifies how many decisions are directly attributable to the group-specific bias in the features. When we remove the bias (add back the artificially subtracted penalty), a substantial fraction of Group 1 individuals cross the decision threshold and would receive positive predictions. This is a direct measurement of the causal effect of the bias on individual outcomes.