Michael Brenndoerfer | Data & AI, Private Equity, Technology

Michael Brenndoerfer

Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Calibration as a Fairness Requirement

Michael BrenndoerferMarch 16, 20264 min read

Part 58 of 67Bias and Fairness

Chapter 444Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Calibration as a Fairness Requirement

Reading section 7 of 13 · Calibration as a Fairness Requirement · 4 minSection 7 of 13 · 4 min

Section 7 of 13 · 4 min read

Fairness Metrics: Demographic Parity, Equalized Odds, and Trade-offs › Calibration as a Fairness RequirementOne section · Sections in this chapter7 / 13 ↓

Sections in this chapter13

Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

As covered in the Calibration chapter from the Evaluation Fundamentals part of this book, a calibrated model produces risk scores that match empirical frequencies. Calibration within groups extends this requirement: among all individuals from group $g$ who receive score $s$ , the actual positive rate should be approximately $s$ .

P(Y = 1 \mid \hat{p} = s, A = g) \approx s \quad \text{for all } g \text{ and } s

where:

$\hat{p}$ is the predicted probability
$s$ is a specific probability value
The condition holds separately for each group $g$

Calibration within groups is a meaningful fairness requirement because it ensures that the score means the same thing for everyone. A predicted probability of 0.7 should mean "70% chance of the positive outcome" for group 0 and group 1 alike. If the model is better calibrated for one group, then the same numerical score carries different information depending on group membership, which can lead to systematically different downstream consequences when decision-makers interpret scores as risk estimates.

The relationship between calibration and predictive parity is important to understand. Predictive parity at a fixed threshold requires equal precision: $P(Y=1 \mid \hat{Y}=1, A=0) = P(Y=1 \mid \hat{Y}=1, A=1)$ . If the model is calibrated within groups (meaning $P(Y=1 \mid \hat{p}=s, A=g) \approx s$ for all $g$ ), then applying the same threshold to both groups will produce the same precision for both groups, achieving predictive parity. Calibration within groups is therefore a sufficient condition for predictive parity. It is a stronger condition because it requires the score-outcome relationship to match at all score levels, not just at the binary threshold.

Let's visualize calibration curves separately for each group to check whether the model is calibrated within groups.

In[20]:

Code

from sklearn.calibration import calibration_curve


def compute_calibration_by_group(y_true, y_prob, group, n_bins=10):
    """
    Compute calibration curves separately for each group.
    Returns fraction of positives and mean predicted probability per bin.
    """
    calibrations = {}
    for g in [0, 1]:
        mask = group == g
        prob_true, prob_pred = calibration_curve(
            y_true[mask], y_prob[mask], n_bins=n_bins, strategy="quantile"
        )
        calibrations[g] = (prob_true, prob_pred)
    return calibrations


cal = compute_calibration_by_group(y_test, y_prob, group_test)

Out[21]:

Visualization

Calibration curves for two demographic groups plotted against perfect calibration diagonal, showing similar curves for both groups. — Calibration curves for Group 0 (blue) and Group 1 (orange), showing the fraction of true positives in each probability bin against the mean predicted probability. The diagonal dashed line represents perfect calibration. Points below the diagonal indicate overconfidence (the model assigns high probabilities but outcomes occur less often than predicted). Group 1's calibration curve tracks Group 0's closely, suggesting the model is roughly calibrated within groups, which is consistent with the near-zero predictive parity difference observed in the fairness metrics.

When calibration curves for both groups track the diagonal closely and track each other, the model is calibrated within groups: the predicted probability means the same thing regardless of group membership. When one group's curve is above the diagonal, the model is under-confident for that group (actual outcomes occur more frequently than predicted). When a curve is below the diagonal, the model is over-confident.

The COMPAS recidivism risk tool controversy is a useful case study here. The tool's developer, Northpointe, showed that among defendants assigned a "high risk" score, the recidivism rate was similar for Black and white defendants, showing calibration within groups. ProPublica showed that the tool assigned high-risk scores to Black defendants who did not reoffend twice as often as white defendants who did not reoffend, showing a violation of equalized odds: specifically, a higher false positive rate for Black defendants. Both findings were mathematically correct. They reflect different fairness definitions applied to the same model, and their apparent contradiction is not accidental. As the impossibility theorem explains, this is exactly what must happen when base rates differ between groups.