Evaluating Survival Predictions › Summary

Evaluating a survival model means scoring two different products with two different instruments. A risk score is evaluated by concordance, which asks whether the model orders failure times correctly on the pairs selected by our strict recorded-time rule: an observed failure at a strictly earlier recorded time than the other unit. Harrell's concordance averages correct orderings over those pairs with ties counted as half, and becomes the ordinary survival concordance when nothing is censored; it is not generally a fixed-horizon binary ROC AUC. Truncated IPCW concordance targets pairs whose earlier failure precedes a supported fixed cutoff. Continuous censoring gives Uno's inverse-square weights; with censoring ties, the implementation uses the corresponding left/right survival product. These weights require the stated censoring assumptions and positive support. A predicted survival curve is evaluated by the Brier score at a horizon, which compares the predicted probability with the realised outcome; the Graf estimator reweights each selected outcome contribution by the reciprocal of its selection probability, so that a unit observed to fail at time $t$ stands in for the $1/\hat{G}(t-)$ similar units represented by that observed failure, and each selected unit observed strictly past $\tau$ stands in for $1/\hat{G}(\tau)$.

The two instruments see different things, so report both when ranking and probability accuracy are decision targets. Concordance uses only the ordering and ties of the supplied scalar risk scores, so it does not identify calibration or the full predicted-curve shape. Changing a curve can change concordance if it changes the chosen risk functional's ordering; a strictly increasing real-number transformation of the supplied scores preserves it. The Brier score is a proper probability score, but its expected value combines reliability (calibration), resolution and marginal outcome uncertainty. Resolution subtracts from that uncertainty, so the uncertainty term is not a lower bound on achievable error. It therefore does not identify calibration by itself. In our equipment cohort the marginal Kaplan–Meier model scored exactly 0.5 on both concordance measures; its Brier scores at the reported horizons answer the separate probability-accuracy question.

Fix the evaluation target before computing the score. Fix the horizons from the decision context before seeing test results, verify that the censoring survival is positive across the evaluation window, use a justified censoring model for the evaluation population, and report intervals rather than points. Never treat a censored duration as an observed failure, never read a c-index as evidence that the probabilities are usable, and justify the censoring model's independence and support assumptions for the population scored. Report the population, horizon, decision, and censoring assumptions alongside the comparison so that readers can interpret the result for that setting.