Evaluating Survival Predictions › Example

List the units with their observed time, event indicator, and risk score: A$(3, 1, 0.90)$, B$(5, 1, 0.60)$, C$(6, 0, 0.70)$, D$(8, 1, 0.30)$, E$(9, 0, 0.40)$, F$(12, 0, 0.10)$. There are $\binom{6}{2} = 15$ unordered pairs, and we classify each one by asking whether the earlier member is an observed failure.

Begin with A. A failed at month 3, so every pair in which A is the earlier member is usable: A with B, C, D, E, and F. That is five pairs, and in every one of them $r_A = 0.90$ exceeds the other score. Five concordant pairs.

B fails at month 5, so the four pairs B vs C, B vs D, B vs E, B vs F are usable. Against C the model is wrong, since $0.60 < 0.70$, giving one discordant pair. Against D, E, F the scores are $0.30, 0.40, 0.10$, all below $0.60$, giving three concordant pairs. Running total: eight concordant, one discordant.

C is censored, so the three pairs C vs D, C vs E, C vs F are excluded. D fails at month 8, so D vs E and D vs F are usable. Against E, $0.30 < 0.40$ is discordant. Against F, $0.30 > 0.10$ is concordant. Running total: nine concordant, two discordant. The remaining pair, E vs F, has both members censored and is excluded.

That gives eleven usable pairs, of which nine are concordant and two discordant, so

$$\hat{C} = \frac{9 + \tfrac{1}{2}\cdot 0}{11} = \frac{9}{11} \approx 0.818.$$

Thus $\hat{C} \approx 0.818$: roughly four times in five, when the data tell us which of two units failed first, the model agrees. Note that the four excluded pairs are not a random sample of the fifteen: they all involve C or E, two of the three censored units, so any claim about the model's accuracy in the late-failure region rests on very little evidence.

For the teaching calculation we fix $\tau^*=10$ months, within positive censoring support. All eleven usable pairs have earlier failures before this cutoff. Their failure times do not tie censorings, so $\hat G(t_i-)=\hat G(t_i)$ and the product weight reduces to $\hat G(t_i)^{-2}$. The censoring survival function comes from the same Kaplan–Meier machinery applied to the censoring indicator:

$$\hat{G}(3) = 1, \quad \hat{G}(5) = 1, \quad \hat{G}(6) = \tfrac{3}{4}, \quad \hat{G}(8) = \tfrac{3}{4}, \quad \hat{G}(9) = \tfrac{3}{8}, \quad \hat{G}(12) = 0.$$

The step at month 6 comes from one censoring event among the four units still under observation, so the curve drops from 1 to $1\cdot(1 - 1/4) = 0.75$. At month 9 one of the two remaining units is censored, so it drops further to $0.75\cdot(1 - 1/2) = 0.375$. At month 12 the last unit is censored and the estimate reaches zero, so the strict Brier implementation used here rejects horizons at or beyond month 12. Censoring exactly at month 12 still establishes survival past that boundary; using that outcome would require the inclusive survivor selection and its matching left-limit censoring weight.

The pair weights are $\hat{G}(t_i)^{-2}$ applied to the earlier member of each pair. A's pairs get weight $1/1^2 = 1$ each, contributing $5$ to the numerator and $5$ to the denominator. B's pairs get $1/1^2 = 1$ each, contributing $3$ to the numerator (three concordant) and $4$ to the denominator (four usable pairs). D's pairs get $1/0.75^2 = 1.7778$ each, contributing $1.7778$ and $3.5556$. Therefore

$$\hat{C}_{\text{Uno}} = \frac{5 + 3 + 1.7778}{5 + 4 + 3.5556} = \frac{9.7778}{12.5556} \approx 0.779.$$

The IPCW version is lower because D's pairs, one of which is discordant, now carry extra weight relative to the others. The two numbers answer slightly different questions: Harrell targets the mix of pairs this study's censoring pattern allowed us to resolve, while truncated IPCW targets true failure pairs whose earlier failure precedes the specified cutoff, under the censoring assumptions.

Compute the Graf contributions in code so every displayed value comes from the data, then check them against the formulas.

Scoring the censored unit as a survivor gives the model unearned credit for an outcome nobody observed, and dropping the censored unit conditions the sample on its horizon outcome being observable, which changes the question being asked. Under the stated censoring assumptions, IPCW targets the complete-outcome Brier error without imputing the unknown outcomes; finite-sample estimation error remains.

The same two calculations, written as explicit loops over pairs, confirm the hand arithmetic and give us a reference to check the vectorised implementation against.