Evaluating Survival Predictions › Advantages

Evaluation metrics that account for censoring let us compare models using the recorded follow-up. A health system triaging transplant candidates, a lender scoring loan prepayment, a maintenance team scheduling overhauls, and a reliability engineer forecasting warranty claims can all face the same observation problem: the outcome of interest is a time, and the observation window ends before every event is observed. The censoring model must be justified for the population scored. Concordance uses observed pairwise orderings. Inverse-probability-of-censoring weighting (IPCW) instead reweights observed contributions under assumptions about censoring and support. Their interpretation depends on those assumptions and on the evaluation horizon; three years of follow-up alone does not establish model quality.

The two families of metrics measure different things, and that separation is a practical advantage. Concordance depends only on the ranking induced by a risk score. It is invariant to any strictly increasing transformation of the score, so it works for Cox linear predictors, for negated AFT log-scale terms, for negated restricted mean survival times, and for the risk output of a random survival forest that never produces an interpretable linear form. The Brier score, by contrast, depends on the predicted probabilities themselves and measures their squared error. Reporting both separates ranking from probability accuracy. Brier error also depends on the outcome uncertainty and resolution, so it does not isolate calibration; the calibration plots below compare predicted and observed probabilities directly.

The Brier score uses an explicit horizon, and truncated IPCW concordance uses a cutoff for the earlier failure; Harrell's concordance here has no horizon parameter. These choices connect evaluation directly to decisions. The prediction we need at a 1,500-hour maintenance window is not the same object as the prediction we need for a five-year warranty reserve, and a horizon-indexed metric can be computed at exactly the horizon the decision uses. Report the scored cohort and evaluation window alongside each metric so that a reader can see which units and period the number describes.

Both metrics accept predictions from different model families. The dense concordance implementation below compares pairs, with quadratic time and memory in the evaluation sample size; the Brier calculation is a mean of per-subject contributions at each horizon. Estimating the censoring distribution adds a Kaplan–Meier fit. Scoring fixed predictions does not refit the survival model, while bootstrapping repeats the scoring calculation. Runtime therefore depends on sample size, horizon count, and the number of resamples.