Random Survival Forests

Michael BrenndoerferOctober 29, 202669 min read

Build random survival forests from scratch: log-rank splits, Nelson-Aalen leaves, censoring-aware risk sets, OOB scoring, and comparison with Cox models.

Machine Learning from ScratchAll chapters
Part 8 of 15Tree-Based Models
Chapter 60Random Survival Forests
Reading the full chapter · 10 sections · 69 min10 sections · 69 min

All 10 sections · 69 min read

Random Survival ForestsFull chapter · Sections in this chapter1 / 10 ↓
Reading tools Definitions, links & notes

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Notes & comments

Random Survival Forests

Consider a cohort of patients discharged after a heart attack. For each patient we record a handful of covariates and the time until readmission. Some patients move to another city, and some are still event-free when the study closes; for those we know only that they were free of readmission up to their last contact. For this right-censored outcome, we record a duration and an indicator showing whether the duration is the event time or a lower bound on it. As we saw in Time-to-Event Data, Censoring, and Hazard Functions, this is right censoring. The model must use those lower bounds without treating them as observed failure times.

The Cox proportional hazards model from Cox Proportional Hazards provides a useful starting point. With a linear predictor and constant coefficients, it models the log hazard ratio as a linear function of the supplied covariates and assumes proportional hazards over time. A threshold effect or an interaction omitted from those covariates can misspecify that predictor. Transformations, splines, and interaction terms can address the problem, but their form must be selected. A survival forest offers a different way to learn nonlinear covariate relationships.

The random forest of Random Forest learns nonlinear relationships for regression and classification by combining trees. Bootstrap samples and random feature subsets vary the trees, while averaging can reduce their prediction variance. Reusing an ordinary regression tree for survival data requires more care: event times hidden by censoring are unavailable for computing the complete-data sample mean and squared-error split criterion. Those population quantities can still exist; the problem is that they are not fully observed.

The random survival forest (RSF) of Ishwaran, Kogalur, Blackstone, and Lauer (2008) adapts the splitting rule and terminal-node prediction to right censoring. We use a log-rank split statistic and a Nelson–Aalen cumulative-hazard estimate in each leaf, retaining bootstrap samples, random feature subsets, and out-of-bag predictions. Our scratch implementation obtains survival by exponentiating the negative ensemble cumulative hazard. Other implementations can use a different survival convention: scikit-survival averages leaf Kaplan–Meier curves for predict_survival_function, while averaging leaf Nelson–Aalen estimates for cumulative-hazard predictions. We will keep those outputs distinct throughout the comparison.

We will build it from scratch in this chapter. First the two components in detail, then a visualization of each, then an eight-patient worked example in which every number can be checked by hand, then a NumPy implementation that we compare with Cox regression and with the reference implementation in scikit-survival.

Advantages

The first advantage is structural. A survival forest does not require a linear log-hazard predictor, and it does not assume that the hazard ratio between any two covariate profiles is constant over time. A tree partitions the covariate space into rectangles and estimates the hazard separately inside each rectangle, so a covariate can matter in one region and not another, effects can be non-monotone, and interactions are discovered rather than specified. The ensemble is not restricted to a common proportional-hazards form, so two covariate profiles may have survival curves that cross, which a Cox model with constant coefficients cannot produce.

The second advantage is that censored observations contribute through risk sets. A leaf's Nelson–Aalen estimate counts each patient until their observed event or censoring time. It does not fill in an unobserved failure time. Interpretation still requires assumptions: independent censoring conditional on covariates is a standard starting point, but a leaf pools several covariate profiles. Conditional independence given the full covariates need not imply independent censoring within that pooled leaf. Finite leaves, adaptive splitting, and heterogeneity can therefore affect estimation; correct risk-set bookkeeping alone does not guarantee unbiased predictions.

The third advantage is practical. Trees compare covariate values with thresholds, so they do not need standardization to make gradient steps well conditioned. Strictly increasing transformations preserve training-value order and the partitions available to an exhaustive threshold search. Non-strict transformations can merge values, and interpolated quantile thresholds, ties, finite precision, and predictions on new values prevent a blanket claim of identical fitted predictions under every monotone map. Out-of-bag predictions provide a useful internal evaluation without holding out each observation from the entire forest fit. They still require care with preprocessing, tuning, censoring, and the evaluation grid.

Disadvantages

One cost is interpretability. A fitted linear Cox model supplies coefficients that describe conditional hazard ratios under its assumptions. A survival forest supplies curves and, often, a single mortality score obtained by summing estimated cumulative hazard over a shared time grid. The score is not a hazard ratio or a probability, and its numerical scale depends on the grid. Permutation importance measures changes in a chosen predictive score; split-based importance measures a different quantity and can favor variables offering many candidate splits. Neither should be interpreted as a treatment effect. Partial dependence and predicted curves can help describe the fitted responses.

Another cost is the amount of information available within each leaf. Log-rank splitting uses event ordering and risk-set counts rather than the spacing between successive event times. Cox partial likelihood also relies on risk sets, so this fact alone does not establish that Cox uses more information or has a smaller mean squared error. A correctly specified proportional-hazards predictor can use its structure effectively, whereas a flexible forest must estimate many local curves. Small event counts and small late risk sets can make those estimates unstable. Event-count and leaf-size constraints help manage that trade-off, without guaranteeing precision.

Survival prediction also requires deliberate conventions. Our forest averages cumulative hazards and then exponentiates the negative average. Averaging the exponentiated tree hazards instead gives a different curve: Jensen's inequality places that average at or above the exponential of the average hazard. This comparison concerns the same tree hazard estimates, not a claim that averaged Kaplan–Meier curves equal either expression. The fitted staircases are flat after their last observed event, which supplies no evidence about later failures. The forest also does not solve informative censoring or automatically correct the consequences of pooling heterogeneous patients.

Formula

We organize the construction around three questions: how to choose a split, what to store at a leaf, and how to combine the trees.

  1. How do we choose a split? A classification tree compares class purity in the two children, and a regression tree compares the spread of the numeric outcome. Their usual complete-data formulas require observed labels or numeric outcomes. Censored event times cannot be inserted as though their observed lower bounds were the outcomes. We need a criterion that uses only what is observed: the ordering of events in time and the sizes of the risk sets on each side of the candidate threshold.
  2. What do we store at a leaf? A regression leaf stores a numeric prediction; a classification leaf stores class probabilities (a probability vector for multiple classes). A survival leaf has to store a function: the survival curve of the patients in the node, estimated from its own risk sets without assuming a parametric shape.
  3. How do we combine the trees? Scikit-learn classification forests average class probabilities, while regression forests average numeric predictions; some classification forests instead combine class votes. A survival forest combines function-valued predictions. Averaging cumulative hazards and averaging survival curves are different choices; either can be meaningful, and averaging alone does not establish unbiasedness or consistency.

Bootstrap sampling, random feature subsets, and recursive axis-aligned partitioning reuse the usual forest scaffold. Event-count stopping rules and out-of-bag scoring also have to account for censored outcomes. We begin with risk sets, derive the split criterion and leaf estimator, then combine the trees.

Notation and risk sets

In the formulas below, "death" denotes the event of interest, including readmission in the opening example.

Everything in the forest is built from the same three quantities observed at each event time: how many patients are still at risk, how many fail, and how the failures are distributed across the children of a split. To write those quantities down, we start with the observation model. Let TiT_i be the observed time for patient ii and δi\delta_i the event indicator, so that δi=1\delta_i = 1 for an event and δi=0\delta_i = 0 for a censored observation. The underlying event time is T~i\tilde T_i and the censoring time is CiC_i, with

Ti=min⁡(T~i,Ci),δi=1{T~i≤Ci}.T_i = \min(\tilde T_i, C_i), \qquad \delta_i = \mathbb{1}\{\tilde T_i \le C_i\}.

The quantities are:

  • TiT_i: the observed time for patient ii, equal to the smaller of the event time and the censoring time
  • T~i\tilde T_i: the underlying (latent) event time for patient ii
  • CiC_i: the censoring time for patient ii
  • δi\delta_i: the event indicator, equal to 1 when the event is observed at or before the censoring time and 0 otherwise
  • 1{⋅}\mathbb{1}\{\cdot\}: the indicator function, equal to 1 when its argument is true and 0 otherwise

The feature vector is Xi∈RpX_i \in \mathbb{R}^p. The central object in everything that follows is the risk set: the set of patients still observable just before time tt. Its size is the number at risk:

R(t)={i:Ti≥t},Y(t)=∣R(t)∣=∑i=1n1{Ti≥t}.R(t) = \{i : T_i \ge t\}, \qquad Y(t) = |R(t)| = \sum_{i=1}^{n} \mathbb{1}\{T_i \ge t\}.

The quantities are:

  • R(t)R(t): the risk set, the set of patients still under observation just before time tt
  • Y(t)Y(t): the number at risk at time tt, the size of the risk set
  • nn: the total number of patients
  • TiT_i: the observed time for patient ii
  • 1{⋅}\mathbb{1}\{\cdot\}: the indicator function, equal to 1 when its argument is true and 0 otherwise

A risk set contains patients whose events could still be observed at time tt. Under the convention Ti≥tT_i \ge t, a patient censored at time 5 contributes at earlier event times and at time 5 itself, then leaves all later risk sets. Dropping that patient at the start would undercount those earlier denominators. Keeping the patient after censoring would overcount later denominators. Risk-set bookkeeping uses the observed follow-up without imputing a later failure time.

The quantity d(t)d(t) is the number of deaths observed exactly at tt:

d(t)=∑i=1n1{Ti=t,δi=1}.d(t) = \sum_{i=1}^{n} \mathbb{1}\{T_i = t, \delta_i = 1\}.

The quantities are:

  • d(t)d(t): the number of deaths observed exactly at time tt
  • TiT_i: the observed time for patient ii
  • δi\delta_i: the event indicator for patient ii, equal to 1 for an event and 0 for a censored observation
  • 1{⋅}\mathbb{1}\{\cdot\}: the indicator function, equal to 1 when its argument is true and 0 otherwise

Grouping events at their recorded times also handles tied observations; it does not require an intrinsically discrete event-time model. At any time tt, we observe the pair (d(t),Y(t))(d(t), Y(t)): how many failed and how many were eligible to fail. For a fixed assignment of patients to candidate children or terminal leaves, these event-time counts determine the log-rank and Nelson–Aalen calculations. The search still needs covariate values and thresholds to form those assignments.

Survival-tree risk sets and split criteria

Now we can state the split criterion. A survival tree recursively partitions covariate space. At each node it compares thresholds on candidate features, forming a left child LL and right child RR. Ordinary squared-error splitting cannot use unobserved failure times as if they were known. Log-rank splitting instead compares the observed event counts with those expected from the two children's risk-set sizes.

What we can compare is what the risk sets tell us at every event time: the fraction of deaths that occur in the left child versus the fraction of patients who were at risk there. Under equal hazards and suitable censoring assumptions, their conditional expectations agree. Observed fractions can differ by chance even for an uninformative split, and crossing hazards can make signed differences cancel. The log-rank statistic sums those observed-minus-expected differences and divides by an aggregate standard deviation. That is the intuition. The formula makes it precise.

For a candidate split, let

YL(t)=∣R(t)∩L∣,YR(t)=∣R(t)∩R∣=Y(t)−YL(t),Y_L(t) = |R(t) \cap L|, \qquad Y_R(t) = |R(t) \cap R| = Y(t) - Y_L(t),

be the number at risk in each child, and let dL(t)d_L(t) be the number of deaths at time tt that fall in the left child, where:

  • R(t)R(t): the risk set at time tt, the set of patients still under observation just before tt
  • LL: the set of patients in the left child of the split
  • RR: the set of patients in the right child of the split (this symbol is overloaded with the risk set R(t)R(t); the argument tt disambiguates the risk set here)
  • YL(t)Y_L(t): the number at risk at time tt in the left child
  • YR(t)Y_R(t): the number at risk at time tt in the right child
  • Y(t)Y(t): the total number at risk at time tt
  • dL(t)d_L(t): the number of deaths at time tt that fall in the left child

If the two children have the same hazard, then the d(t)d(t) deaths at time tt are allocated between the children in proportion to how many people were at risk there, so the expected number of deaths in the left child is

E[dL(t)]=d(t) YL(t)Y(t).\mathbb{E}[d_L(t)] = d(t)\,\frac{Y_L(t)}{Y(t)}.

The quantities are:

  • E[dL(t)]\mathbb{E}[d_L(t)]: the expected number of deaths in the left child at time tt under the null of equal hazards
  • d(t)d(t): the total number of deaths at time tt
  • YL(t)Y_L(t): the number at risk at time tt in the left child
  • Y(t)Y(t): the total number at risk at time tt

The observed number minus the expected number,

dL(t)−d(t) YL(t)Y(t),d_L(t) - d(t)\,\frac{Y_L(t)}{Y(t)},

is the observed-minus-expected event contribution at that time. The log-rank numerator sums these contributions, and its denominator is the square root of their summed null variances. It does not standardize each event-time contribution separately. The variance calculation below accounts for the number of deaths and the allocation of patients between the two risk sets.

The probability of exactly kk deaths in LL when d(t)d(t) deaths are drawn without replacement from a population of Y(t)Y(t) with YL(t)Y_L(t) members is hypergeometric, and its variance is

V(t)=d(t) (Y(t)−d(t)) YL(t) YR(t)Y(t)2 (Y(t)−1).V(t) = \frac{d(t)\,\bigl(Y(t) - d(t)\bigr)\,Y_L(t)\,Y_R(t)}{Y(t)^2\,\bigl(Y(t) - 1\bigr)}.

The quantities are:

  • V(t)V(t): the hypergeometric variance of the number of deaths in the left child at time tt
  • d(t)d(t): the total number of deaths at time tt
  • Y(t)Y(t): the total number at risk at time tt
  • YL(t)Y_L(t): the number at risk at time tt in the left child
  • YR(t)Y_R(t): the number at risk at time tt in the right child

Write pL=YL(t)/Y(t)p_L=Y_L(t)/Y(t). For Y(t)>1Y(t)>1, the same hypergeometric variance is d(t)pL(1−pL)[Y(t)−d(t)]/[Y(t)−1]d(t)p_L(1-p_L)[Y(t)-d(t)]/[Y(t)-1]. The first factor is the binomial variance for d(t)d(t) draws, and the last factor corrects for sampling without replacement. An empty child or an event count equal to the whole risk set gives zero variance. When Y(t)=1Y(t)=1, the displayed fraction is undefined; the implementation assigns zero because no between-child comparison remains.

Summing the surprises and the variances over all event times gives a single standardized statistic. Writing t1<t2<…t_1 < t_2 < \dots for the distinct event times,

L(s)=∑j(dL(tj)−d(tj) YL(tj)Y(tj))∑jV(tj),L(s) = \frac{\sum_{j} \left( d_L(t_j) - d(t_j)\,\dfrac{Y_L(t_j)}{Y(t_j)} \right)}{\sqrt{\sum_{j} V(t_j)}},

The quantities are:

  • L(s)L(s): the standardized log-rank statistic for candidate split ss
  • ss: the index of the candidate split
  • tjt_j: the jj-th distinct event time
  • dL(tj)d_L(t_j): the observed number of deaths in the left child at time tjt_j
  • d(tj)d(t_j): the total number of deaths at time tjt_j
  • YL(tj)Y_L(t_j): the number at risk at time tjt_j in the left child
  • Y(tj)Y(t_j): the total number at risk at time tjt_j
  • V(tj)V(t_j): the hypergeometric variance of the left-child death count at time tjt_j

The tree picks the split that maximizes ∣L(s)∣|L(s)|. A positive sign indicates more observed left-child events than expected in the summed comparison; it need not mean a larger hazard at every time. For a prespecified split, the standardized statistic has an approximately standard-normal null distribution under the usual log-rank central-limit conditions, including independent censoring, growing nondegenerate event information, and no dominating event-time contribution. With untied failures it is the standardized Cox score at β=0\beta=0 for the binary group indicator. Tied-failure conventions can change the Cox information calculation, so that exact equivalence needs its convention stated.

This is a greedy search, and it is biased. The maximum of a large collection of statistics is not itself a standard normal, and features with many distinct values offer more candidate thresholds and therefore more chances to find a large maximum. Because we maximize over thresholds, we should not turn the split value into a pp-value. The random survival forest mitigates the problem by considering only a random subset of features at each split and by requiring a minimum number of events in each child, but the mitigation is not a cure.

Terminal-node survival and hazard estimates

The split criterion gives us a partition, but a partition is only useful if we can describe the survival behavior of each cell. This is the second piece of machinery. Once a node is declared terminal, we need to turn its patients into a survival curve, and we want the curve to assume no parametric shape. The natural starting point is the nonparametric estimator from the counting-process chapter, the Nelson–Aalen estimator. For a terminal node hh with event times th,1<⋯<th,mht_{h,1} < \dots < t_{h,m_h}, the number of deaths at th,jt_{h,j} is dh,jd_{h,j} and the number at risk just before it is Yh,jY_{h,j}. The estimated cumulative hazard is

H^h(t)=∑j : th,j≤tdh,jYh,j,\hat{H}_h(t) = \sum_{j\,:\,t_{h,j} \le t} \frac{d_{h,j}}{Y_{h,j}},

The quantities are:

  • H^h(t)\hat{H}_h(t): the Nelson–Aalen estimate of the cumulative hazard in node hh at time tt
  • hh: the terminal node
  • th,jt_{h,j}: the jj-th distinct event time in node hh
  • dh,jd_{h,j}: the number of deaths at th,jt_{h,j} in node hh
  • Yh,jY_{h,j}: the number at risk just before th,jt_{h,j} in node hh

and the implied survival function is S^h(t)=exp⁡(−H^h(t))\hat S_h(t) = \exp(-\hat H_h(t)).

Compare this estimate with the Kaplan–Meier product S^KM,h(t)=∏j:th,j≤t(1−dh,j/Yh,j)\hat S_{\mathrm{KM},h}(t)=\prod_{j:t_{h,j}\le t}(1-d_{h,j}/Y_{h,j}). Each factor is the empirical conditional survival fraction at an event time. Where every increment u=d/Yu=d/Y is small, the expansion −log⁡(1−u)=u+O(u2)-\log(1-u)=u+O(u^2) explains why the negative log of Kaplan–Meier is close to Nelson–Aalen. They need not be equal in a finite leaf. If the last remaining patient fails, Kaplan–Meier reaches zero while exp⁡(−H^h)\exp(-\hat H_h) remains positive. Both are valid finite-follow-up survival estimates; neither flat tail identifies the unobserved lifetime distribution beyond follow-up.

The terminal node therefore stores a step function, not a number. Its jumps are at the event times of the node, and its height is determined by the node's own risk sets. Jump sizes depend on d/Yd/Y at each event time. A small late risk set can create a large jump even when earlier risk sets were large. This makes event counts and risk-set sizes useful tuning considerations, and it is why the forest's terminal nodes are not "pure" in the sense of classification trees: they are small samples from which a cumulative hazard can be estimated.

A second estimate, the "mortality" or risk score, is used to rank patients and is a summary of the whole step function. It exists because ranking and reporting often demand a single number per patient, even though the primary output is a curve. With a common grid t1<⋯<tKt_1 < \dots < t_K of times shared by all trees,

τ(x)=∑k=1KH^(tk∣x),\tau(x) = \sum_{k=1}^{K} \hat H(t_k \mid x),

The quantities are:

  • τ(x)\tau(x): the risk score for a patient with covariates xx, the sum of the ensemble cumulative hazard over the grid
  • xx: the covariate vector of the patient
  • t1<⋯<tKt_1 < \dots < t_K: the common time grid shared by all trees
  • KK: the number of points on the common grid
  • H^(tk∣x)\hat H(t_k \mid x): the ensemble cumulative hazard at time tkt_k for covariates xx

which is the sum of the ensemble cumulative hazard over the grid, i.e. the average across trees of the total hazard accumulated over the grid. It is a rank-order score and it is not a probability. A larger τ\tau means a larger sum on this grid, but does not imply larger cumulative hazard or lower survival at every time when curves cross; two patients whose scores differ by a factor of two are simply in a different order, not "twice as sick".

Bootstrap ensembles and out-of-bag predictions

The third piece is the ensemble. A single tree can have high sampling variance: its splits depend on the training sample, and a small change in the data can move a threshold and change predictions below it. The ensemble can reduce that variance by averaging trees with imperfectly correlated predictions. Bootstrap samples provide diversity and omitted observations for OOB assessment, subject to the evaluation caveats below.

The forest is a collection of trees, each grown on a bootstrap sample of the observations. Let H^b(t∣x)\hat H_b(t \mid x) be the cumulative hazard for the bb-th tree, evaluated at a point xx by pushing xx down the tree and returning the step function of the terminal node it reaches. The ensemble estimate is

H^(t∣x)=1B∑b=1BH^b(t∣x),S^(t∣x)=exp⁡ ⁣(−H^(t∣x)).\hat H(t \mid x) = \frac{1}{B} \sum_{b=1}^{B} \hat H_b(t \mid x), \qquad \hat S(t \mid x) = \exp\!\left(-\hat H(t \mid x)\right).

The quantities are:

  • H^(t∣x)\hat H(t \mid x): the ensemble cumulative hazard at time tt for covariates xx
  • BB: the number of trees in the forest
  • H^b(t∣x)\hat H_b(t \mid x): the cumulative hazard of the bb-th tree at time tt for covariates xx
  • S^(t∣x)\hat S(t \mid x): the ensemble survival function at time tt for covariates xx

Our scratch forest uses the cumulative-hazard average shown above and defines survival as its negative exponential. For any fixed collection of tree hazards, convexity gives B−1∑bexp⁡(−H^b)≥exp⁡(−B−1∑bH^b)B^{-1}\sum_b\exp(-\hat H_b)\ge\exp(-B^{-1}\sum_b\hat H_b). Both sides are nonincreasing curves in [0,1][0,1], but they need not agree. This finite-ensemble inequality supplies no automatic unbiasedness or consistency result. Scikit-survival's cumulative-hazard method also averages Nelson–Aalen estimates; its survival method instead averages leaf Kaplan–Meier estimates. Those survival curves are not the left-hand expression unless the leaf estimators happen to coincide. Check the actual prediction method before comparing implementations.

The following deterministic toy ensemble isolates that gap. Five nondecreasing cumulative-hazard curves share one baseline shape but differ in scale, so the difference between the two aggregation conventions is not confounded by sampling noise.

VisualizationOut [4]
CollapseExpand
Survival from averaged hazards lies below the average of tree survival curves.
Comparison of two ways to combine an ensemble of survival trees. The blue curve exponentiates the average cumulative hazard, while the orange curve averages the individual survival functions; the shaded band is the Jensen gap, nonnegative because the exponential is convex. The two conventions differ systematically, so the aggregation choice should be reported before scores are compared across implementations.

The bootstrap also supplies the out-of-bag estimate. Observation ii is out of bag for tree bb if the bb-th bootstrap sample did not contain it. Let Bi={b:i∉bootstrap(b)}B_i = \{b : i \notin \text{bootstrap}(b)\} be the set of trees for which ii is out of bag. The probability that a given observation is in a given bootstrap sample is 1−(1−1/n)n→1−e−1≈0.6321 - (1 - 1/n)^n \to 1 - e^{-1} \approx 0.632, so its exact expected size is B(1−1/n)nB(1-1/n)^n, approximately 0.368B0.368B for large nn. The out-of-bag prediction is

H^OOB(t∣xi)=1∣Bi∣∑b∈BiH^b(t∣xi),\hat H_{\mathrm{OOB}}(t \mid x_i) = \frac{1}{|B_i|} \sum_{b \in B_i} \hat H_b(t \mid x_i),

The quantities are:

  • H^OOB(t∣xi)\hat H_{\mathrm{OOB}}(t \mid x_i): the out-of-bag cumulative hazard at time tt for observation ii
  • xix_i: the covariate vector of observation ii
  • BiB_i: the set of trees for which observation ii is out of bag
  • ∣Bi∣|B_i|: the number of such trees
  • H^b(t∣xi)\hat H_b(t \mid x_i): the cumulative hazard of tree bb at time tt for observation ii

Here only trees whose bootstrap samples exclude observation ii contribute to its prediction. If ∣Bi∣=0|B_i|=0, that prediction is unavailable. This avoids training those trees' splits and leaves on the held-out observation, but it does not make the complete OOB procedure independent of that observation: this implementation uses the full training event-time grid when forming mortality scores. Shared preprocessing or repeated OOB tuning can introduce further dependencies. Treat OOB concordance as an internal estimate, and reserve an independent test set for a final comparison.

Mathematical Properties

Three properties summarize what we have built, and each points to a different reason to be careful when the forest is applied.

For a prespecified binary grouping with untied failures, the log-rank comparison agrees with the standardized Cox score at zero coefficient. This connects the split criterion to a local proportional-hazards comparison without imposing proportional hazards on the fitted forest. Choosing the largest statistic over many thresholds changes its null distribution, and crossing hazards can reduce the power of the signed comparison.

The Nelson–Aalen estimator adds the non-negative increments dh,j/Yh,jd_{h,j}/Y_{h,j}. At a fixed time tt, a commonly used variance estimate is ∑j:th,j≤tdh,j/Yh,j2\sum_{j:t_{h,j}\le t}d_{h,j}/Y_{h,j}^2. This estimates uncertainty in the cumulative-hazard staircase; it is distinct from the hypergeometric variance used in the log-rank split statistic.

Averaging can reduce variance; it does not in itself remove bias. If random tree predictions at a fixed (t,x)(t,x) have common variance σ2\sigma^2 and common pairwise correlation ρ\rho, the covariance sum gives Var⁡(B−1∑bH^b)=ρσ2+(1−ρ)σ2/B\operatorname{Var}(B^{-1}\sum_b\hat H_b)=\rho\sigma^2+(1-\rho)\sigma^2/B. This is an exchangeable-prediction model, not a claim that trees are unbiased. Independently randomized trees can be conditionally independent given the training data while remaining dependent across repeated training datasets. More trees reduce the remaining randomization variation, with diminishing gains; feature subsampling can change both correlation and individual-tree strength.

The variance formula can be read directly from a small curve. For a fixed unit individual-prediction variance, the floor is the tree correlation, and more trees cannot remove variance that the trees share.

VisualizationOut [6]
CollapseExpand
Variance falls with more trees and plateaus at a level set by tree correlation.
Variance of the ensemble average as a function of forest size for several tree correlations. With independent trees the curve decays like one over B, while with correlated trees it levels off at a floor of rho times the individual-tree variance under this covariance model. Additional trees still reduce the remaining one-over-B term, with diminishing gains.

The predictions are step functions with jumps at the union of the training event times, so they have jumps rather than a smooth time derivative and supply no learned extrapolation beyond the largest observed event time. At that point the cumulative hazard is flat, the survival function is constant, and the model is silent about what happens next.

Visualizing Random Survival Forests

Before fitting anything, we should see how the two ingredients of the model work. We simulate a dataset in which the truth is known, and use it to look at the split search and at the terminal-node estimate.

The data generator is a Weibull model in which the log hazard is a nonlinear function of the covariates. The Weibull generator uses a shape parameter and a rate parameter. We distinguish the generator parameters from the earlier tree-correlation symbol ρ\rho: the Weibull shape parameter is a=1.5a = 1.5 and the Weibull rate parameter is b=0.2b = 0.2, so the conditional survival is S(t∣x)=exp⁡(−b taeη(x))S(t \mid x) = \exp(-b\, t^{a} e^{\eta(x)}). The covariates are five independent standard normal variables, the first two of which are informative and the last three of which are noise. The covariate contribution to the log hazard is

η(x)=1.2 x1+1.2 sin⁡(2x1)+1.2 (x22−1),\eta(x) = 1.2\,x_1 + 1.2\,\sin(2 x_1) + 1.2\,(x_2^2 - 1),

which contains a linear term, a periodic term, and a quadratic term, and therefore a linear model in x1x_1 and x2x_2 cannot represent the truth. Censoring is exponential with rate 0.300.30, which produces roughly half of the observations censored at the generated event-time scale.

The quantities are:

  • x1,x2,…,x5x_1, x_2, \ldots, x_5: the five standard-normal covariates, of which only x1x_1 and x2x_2 enter η\eta
  • η(x)\eta(x): the covariate contribution to the log hazard; the Weibull baseline also depends on time
  • aa: the Weibull shape parameter, 1.51.5 here
  • bb: the Weibull rate parameter, 0.20.2 here
  • S(t∣x)S(t \mid x): the conditional survival function of the event time given the covariates

The simulated censoring times are independent of both the covariates and the latent event times. This gives independent censoring in this simulation.

PythonIn [7]
CollapseExpand
def make_survival_data(n, rng, nonlinear=True):
    """Simulate Weibull survival times with a nonlinear log hazard."""
    X = rng.normal(size=(n, 5))
    if nonlinear:
        eta = (
            1.2 * X[:, 0]
            + 1.2 * np.sin(2.0 * X[:, 0])
            + 1.2 * (X[:, 1] ** 2 - 1.0)
        )
    else:
        eta = 1.0 * X[:, 0] + 0.8 * X[:, 1]
    shape, rate = (
        1.5,
        0.20,
    )  # named a and b in the text to avoid clashing with rho and lambda
    u = rng.uniform(size=n)
    event_time = (-np.log(u) / (rate * np.exp(eta))) ** (1.0 / shape)
    censor_time = rng.exponential(scale=1.0 / 0.30, size=n)
    return (
        X,
        np.minimum(event_time, censor_time),
        event_time <= censor_time,
        eta,
    )


rng = np.random.default_rng(20240517)
X_train, t_train, e_train, eta_train = make_survival_data(500, rng)
X_test, t_test, e_test, eta_test = make_survival_data(400, rng)
ConsoleOut [8]
CollapseExpand
training rows: 500, events: 246, censored: 50.8%
test rows:     400, events: 186, censored: 53.5%
median follow-up (training): 0.96

The event counts, censoring percentages, and median follow-up above are computed from the simulated arrays and can vary with the seed. The requested row counts remain 500 for training and 400 for testing. In this simulated example, a few hundred events allow several leaves under the minimum-events rule, while censoring makes the risk-set bookkeeping visible.

The first ingredient is the log-rank statistic. The function below implements the formula we derived, with the sorted arrays used to compute the risk sets at each distinct event time. The grouping vector is true for the left child and false for the right child; a negative returned statistic indicates fewer left-child events than expected in the summed comparison, and its absolute value is what the split search maximizes.

PythonIn [9]
CollapseExpand
def logrank_statistic(time, event, group):
    """Standardised log-rank statistic for a two-group split."""
    time = np.asarray(time, dtype=float)
    event = np.asarray(event, dtype=bool)
    group = np.asarray(group, dtype=bool)
    n = time.size

    order = np.argsort(time, kind="mergesort")
    t_sorted = time[order]
    e_sorted = event[order].astype(float)
    g_sorted = group[order].astype(float)

    unique_times, first = np.unique(t_sorted, return_index=True)
    at_risk = n - first
    deaths = np.add.reduceat(e_sorted, first)
    deaths_g = np.add.reduceat(e_sorted * g_sorted, first)

    group_before = np.concatenate(([0.0], np.cumsum(g_sorted)))[first]
    at_risk_g = g_sorted.sum() - group_before
    at_risk_o = at_risk - at_risk_g

    keep = deaths > 0
    d, d_g = deaths[keep], deaths_g[keep]
    y, y_g, y_o = at_risk[keep], at_risk_g[keep], at_risk_o[keep]

    observed = d_g
    expected = y_g * d / y
    variance = np.zeros_like(d)
    ok = y > 1
    variance[ok] = (
        d[ok]
        * (y[ok] - d[ok])
        * y_g[ok]
        * y_o[ok]
        / (y[ok] ** 2 * (y[ok] - 1.0))
    )

    total_variance = variance.sum()
    if total_variance <= 0:
        return 0.0
    return float((observed - expected).sum() / np.sqrt(total_variance))

The keep filter is important: the statistic is a sum over event times, because a time at which nobody dies contributes nothing to either the numerator or the variance. The ok guard handles any event time at which the risk set has only a single person, where the displayed variance fraction is undefined and the comparison variance is zero.

The split search sweeps over candidate thresholds. For each feature we take a grid of quantiles, form the group membership, and evaluate the statistic. This is deliberately the same kind of search a tree performs, restricted here to a coarse grid.

PythonIn [10]
CollapseExpand
def best_split(X, time, event, features, n_cuts=10, min_leaf=15, min_events=8):
    """Search a quantile grid on each candidate feature; return the best log-rank split."""
    best = None
    for feature in features:
        x = X[:, feature]
        for cut in np.unique(np.quantile(x, np.linspace(0.1, 0.9, n_cuts))):
            left = x <= cut
            n_left = int(left.sum())
            if n_left < min_leaf or x.size - n_left < min_leaf:
                continue
            d_left = event[left].sum()
            if d_left < min_events or event.sum() - d_left < min_events:
                continue
            stat = logrank_statistic(time, event, left)
            if best is None or abs(stat) > abs(best[2]):
                best = (int(feature), float(cut), stat)
    return best

The two figures below sweep thresholds on the two informative features and plot the signed log-rank statistic. On this seeded sample, the x1x_1 curve stays negative and reaches its largest magnitude at an interior threshold near 0.530.53. The x2x_2 curve changes sign near the centre, with larger magnitudes toward the tails. Its largest magnitude on this grid occurs near 1.561.56, separating the upper tail from the remaining observations. The split search compares absolute values, not the heights of the signed curves.

PythonIn [11]
CollapseExpand
candidates_x1 = np.quantile(X_train[:, 0], np.linspace(0.02, 0.98, 60))
candidates_x2 = np.quantile(X_train[:, 1], np.linspace(0.02, 0.98, 60))
stats_x1 = np.array(
    [
        logrank_statistic(t_train, e_train, X_train[:, 0] <= c)
        for c in candidates_x1
    ]
)
stats_x2 = np.array(
    [
        logrank_statistic(t_train, e_train, X_train[:, 1] <= c)
        for c in candidates_x2
    ]
)
PythonIn [12]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, theme_color, use_book_style


def pcol(name, fallback=0):
    try:
        return PALETTE[name]
    except (KeyError, TypeError, IndexError):
        return plt.rcParams["axes.prop_cycle"].by_key()["color"][fallback]


use_book_style()

plt.figure(figsize=(6.0, 4.0))
ax = plt.gca()
ax.plot(candidates_x1, stats_x1, color=PALETTE["primary"], linewidth=1.1)
ax.axhline(0.0, color=PALETTE["muted"], linewidth=0.8, linestyle=":")
k = int(np.argmax(np.abs(stats_x1)))
ax.plot(
    candidates_x1[k],
    stats_x1[k],
    "o",
    markerfacecolor=PALETTE["paper"],
    markeredgecolor=PALETTE["primary"],
    markersize=8,
)
ax.axvline(
    candidates_x1[k], color=PALETTE["primary"], linestyle="--", linewidth=1.0
)
ax.set_xlabel("candidate threshold on $x_1$")
ax.set_ylabel("log-rank statistic $L(s)$")
ax.set_title("Split search on the linear-plus-sine feature")
polish_axes(ax)
plt.show()
VisualizationOut [12]
CollapseExpand
A line plot of the signed log-rank statistic against candidate thresholds on x1. The curve stays below zero and has a broad interior trough, and an open marker with a dashed vertical line shows the selected threshold.
Log-rank statistic across every candidate threshold on x1. Each point is one threshold from a quantile grid, and the open marker with a dashed line marks the value a tree would select. The curve stays negative and has a broad interior trough, with the selected threshold near the minimum, showing that the split search compares absolute values rather than signed heights.
PythonIn [13]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, theme_color, use_book_style


def pcol(name, fallback=0):
    try:
        return PALETTE[name]
    except (KeyError, TypeError, IndexError):
        return plt.rcParams["axes.prop_cycle"].by_key()["color"][fallback]


use_book_style()

plt.figure(figsize=(6.0, 4.0))
ax = plt.gca()
ax.plot(candidates_x2, stats_x2, color=PALETTE["amber"], linewidth=1.1)
ax.axhline(0.0, color=PALETTE["muted"], linewidth=0.8, linestyle=":")
k = int(np.argmax(np.abs(stats_x2)))
ax.plot(
    candidates_x2[k],
    stats_x2[k],
    "o",
    markerfacecolor=PALETTE["paper"],
    markeredgecolor=PALETTE["amber"],
    markersize=8,
)
ax.axvline(
    candidates_x2[k], color=PALETTE["amber"], linestyle="--", linewidth=1.0
)
ax.set_xlabel("candidate threshold on $x_2$")
ax.set_ylabel("log-rank statistic $L(s)$")
ax.set_title("Split search on the quadratic feature")
polish_axes(ax)
plt.show()
VisualizationOut [13]
CollapseExpand
A line plot of the signed log-rank statistic against candidate thresholds on x2. The curve moves from positive to negative and reaches its largest absolute value in the upper tail, and an open marker with a dashed vertical line shows the selected threshold.
Log-rank statistic across every candidate threshold on x2. Each point is one threshold from a quantile grid, and the open marker with a dashed line marks the value a tree would select. The curve changes sign near the center and reaches its largest magnitude in the upper tail, where the selected threshold isolates the high-risk tail and leaves the opposite tail in the other child.

The x1x_1 curve has a broad interior trough rather than a monotone shape. The selected threshold near x1=0.53x_1=0.53 separates a lower-risk left child from a higher-risk right child in this sample. The x2x_2 curve shows a threshold near x2=1.56x_2=1.56 that isolates the upper high-risk tail. A single threshold cannot put both tails of a quadratic effect into one child while leaving the centre in the other. Later splits can isolate the two tails in separate leaves, allowing their predictions to differ from the centre. Bootstrap samples and random feature subsets can change the selected thresholds across trees.

What a terminal node stores

The second ingredient is the Nelson–Aalen estimator. The function below sorts the data, groups the times, and accumulates the increments. It returns the distinct event times and the cumulative hazard at each of them, so the result is a step function that can be evaluated anywhere.

PythonIn [14]
CollapseExpand
def nelson_aalen(time, event):
    """Nelson-Aalen estimate of the cumulative hazard: event times and cumulative hazard."""
    time = np.asarray(time, dtype=float)
    event = np.asarray(event, dtype=bool)
    n = time.size
    if n == 0:
        return np.empty(0), np.empty(0)

    order = np.argsort(time, kind="mergesort")
    t_sorted, e_sorted = time[order], event[order]

    unique_times, first = np.unique(t_sorted, return_index=True)
    at_risk = n - first
    deaths = np.add.reduceat(e_sorted.astype(float), first)

    keep = deaths > 0
    return unique_times[keep], np.cumsum(deaths[keep] / at_risk[keep])

The at_risk = n - first line is the whole risk-set computation in disguise: first is the index of the first occurrence of each distinct time in the sorted arrays, so n - first is the number of observations whose time is at least that value. It is the same as counting the patients with Ti≥tT_i \ge t, and it takes linear time after sorting, in addition to the sorting cost.

Now we take the best split found on the full data, form the two children, and estimate the cumulative hazard in each. The visual sweeps above use a finer, wider quantile grid than this helper; here the root search also enforces at least 25 observations and 15 events in each child, so its selected threshold can differ from the sweep markers.

PythonIn [15]
CollapseExpand
root_feature, root_cut, root_stat = best_split(
    X_train,
    t_train,
    e_train,
    features=range(X_train.shape[1]),
    min_leaf=25,
    min_events=15,
)
in_left = X_train[:, root_feature] <= root_cut
times_all, cumhaz_all = nelson_aalen(t_train, e_train)
times_left, cumhaz_left = nelson_aalen(t_train[in_left], e_train[in_left])
times_right, cumhaz_right = nelson_aalen(t_train[~in_left], e_train[~in_left])
plot_times_all = np.r_[0.0, times_all]
plot_cumhaz_all = np.r_[0.0, cumhaz_all]
plot_times_left = np.r_[0.0, times_left]
plot_cumhaz_left = np.r_[0.0, cumhaz_left]
plot_times_right = np.r_[0.0, times_right]
plot_cumhaz_right = np.r_[0.0, cumhaz_right]
PythonIn [16]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, use_book_style

use_book_style()

plt.figure(figsize=(6.0, 4.6))
ax = plt.gca()
ax.step(
    plot_times_all,
    plot_cumhaz_all,
    where="post",
    color=PALETTE["muted"],
    linewidth=1.1,
    label="all patients",
)
ax.step(
    plot_times_left,
    plot_cumhaz_left,
    where="post",
    color=PALETTE["primary"],
    linewidth=1.1,
    label=f"$x_{{{root_feature + 1}}} \\leq {root_cut:.2f}$",
)
ax.step(
    plot_times_right,
    plot_cumhaz_right,
    where="post",
    color=PALETTE["amber"],
    linewidth=1.1,
    label=f"$x_{{{root_feature + 1}}} > {root_cut:.2f}$",
)
ax.set_xlabel("time")
ax.set_ylabel("cumulative hazard $\\hat{H}(t)$")
ax.set_xlim(0.0, 6.0)
ax.set_title("Nelson–Aalen cumulative hazard by child")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.20), ncol=3)
polish_axes(ax)
plt.show()
VisualizationOut [16]
CollapseExpand
Three cumulative hazard step functions: a muted reference curve for all patients, a blue curve for the low-risk child that stays low, and an orange curve for the high-risk child that climbs steeply.
Nelson–Aalen cumulative hazard staircases for the two children of a single split. The muted reference curve is the whole sample, the blue curve is the left child, and the orange curve is the right child. The children separate immediately, showing that each terminal node stores a step function rather than a single number, with jump sizes set by the node's own risk sets.
PythonIn [17]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, use_book_style

use_book_style()

plt.figure(figsize=(6.0, 4.6))
ax = plt.gca()
ax.step(
    plot_times_all,
    np.exp(-plot_cumhaz_all),
    where="post",
    color=PALETTE["muted"],
    linewidth=1.1,
    label="all patients",
)
ax.step(
    plot_times_left,
    np.exp(-plot_cumhaz_left),
    where="post",
    color=PALETTE["primary"],
    linewidth=1.1,
    label=f"$x_{{{root_feature + 1}}} \\leq {root_cut:.2f}$",
)
ax.step(
    plot_times_right,
    np.exp(-plot_cumhaz_right),
    where="post",
    color=PALETTE["amber"],
    linewidth=1.1,
    label=f"$x_{{{root_feature + 1}}} > {root_cut:.2f}$",
)
ax.set_xlabel("time")
ax.set_ylabel("survival $\\hat{S}(t) = \\exp(-\\hat{H}(t))$")
ax.set_xlim(0.0, 6.0)
ax.set_title("Survival is the exponential of the negative")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.20), ncol=3)
polish_axes(ax)
plt.show()
VisualizationOut [17]
CollapseExpand
Three survival step curves that start at one: a blue curve for the low-risk child that declines slowly, an orange curve for the high-risk child that declines quickly, and a muted reference curve for all patients in between.
Implied survival functions for the two children of a single split, obtained by exponentiating the negative of the Nelson–Aalen cumulative hazard. The high-risk child declines faster than the low-risk child, and the whole-sample curve lies between them, reflecting the same partition seen in the cumulative-hazard figure.

The fitted child staircases separate in this sample; a selected split can also separate curves by chance. The jumps are visible: each event time in a node produces a vertical increment of size d/Yd/Y, and their sizes depend on the local event count and number still at risk. A minimum event count does not itself bound the last jump. If a leaf had only one event and one patient, the jump would be a full unit of hazard, and the survival function would collapse to e−1e^{-1} in one step. Our implementation constrains both observations and events per child. Other implementations, including the reference used below, express their leaf-size constraints differently.

Example

The formulas are easier to trust once every number has been computed by hand. We use eight patients and a single binary covariate, so that the split is given and the arithmetic is the only thing left to do. Every quantity we compute below maps to one of the equations above: the risk sets and counts feed the split criterion, and the same counts feed the terminal-node estimate.

Group A is the low-risk group: four patients, of whom one has an event at time 8 and three are censored at times 1, 2 and 3. Group B is the high-risk group: four patients with events at times 1, 2, 3 and 4. The table below is the entire dataset.

PythonIn [18]
CollapseExpand
example = pd.DataFrame(
    {
        "patient": np.arange(1, 9),
        "group": ["A"] * 4 + ["B"] * 4,
        "time": [1, 2, 3, 8, 1, 2, 3, 4],
        "event": [0, 0, 0, 1, 1, 1, 1, 1],
    }
)
example["role"] = np.where(example["event"] == 1, "event", "censored")
Out[19]:
patient group time event role
0 1 A 1 0 censored
1 2 A 2 0 censored
2 3 A 3 0 censored
3 4 A 8 1 event
4 5 B 1 1 event
5 6 B 2 1 event
6 7 B 3 1 event
7 8 B 4 1 event

The split ss puts group A on the left and group B on the right. We compute the log-rank statistic at each distinct event time. There are five of them: 1, 2, 3, 4 and 8. At time tt, the risk set is everybody with a time at least tt, whether censored or not, because a censoring does not remove a patient from the risk set before their own observed time. Every number in the table below can be reconstructed from the patient table above by applying the definitions from the notation subsection one event time at a time.

Log-rank contributions at each distinct event time for the two-group split.
ttY(t)Y(t)d(t)d(t)YL(t)Y_L(t)dL(t)d_L(t)EL(t)E_L(t)V(t)V(t)
181400.500.250
261300.500.250
341200.500.250
421100.500.250
811111.000.000

At t=1t=1 the risk set is all eight patients, one of whom dies, and four of the eight are in group A, so the expected number of deaths in A is 1×4/8=0.51 \times 4/8 = 0.5. The variance is 1×7×4×4/(82×7)=0.251 \times 7 \times 4 \times 4 / (8^2 \times 7) = 0.25. The censoring at times 1, 2, and 3 does not remove those patients from the risk set until their own observed time: a patient censored at time 3 is still at risk for the events at times 1 and 2, and remains at risk for an event tied at time 3, leaving the risk set after that time. That is why the risk set shrinks gradually as the table progresses rather than dropping by the number of censored patients at each step, and it is why the arithmetic is easier to read from the table than from the change.

At t=8t=8 the risk set has a single patient, and that patient is the only one in group A with an event. The observed number of deaths in A is 1 and the expected number is 1, so the contribution to the numerator is zero. The displayed variance fraction would divide by zero; the comparison variance is assigned zero because the sole patient must have that event: no comparison between groups is possible when only one patient remains. The time contributes nothing, which is the correct behaviour.

Summing the columns gives the statistic:

∑j(dL(tj)−EL(tj))=0−0.5−0.5−0.5−0.5+0=−2.0,\sum_j \bigl(d_L(t_j) - E_L(t_j)\bigr) = 0 - 0.5 - 0.5 - 0.5 - 0.5 + 0 = -2.0, ∑jV(tj)=0.25+0.25+0.25+0.25=1.0,\sum_j V(t_j) = 0.25 + 0.25 + 0.25 + 0.25 = 1.0, L(s)=−2.01.0=−2.0.L(s) = \frac{-2.0}{\sqrt{1.0}} = -2.0.

The negative sign indicates fewer group-A events than expected in the log-rank comparison. The squared statistic is 4.04.0. For a prespecified grouping, its asymptotic one-degree-of-freedom chi-square tail probability is about 0.0460.046. With only eight patients and five events, that approximation should not be treated as reliable small-sample significance; it would also be inappropriate after searching many candidate splits.

Now we estimate the two terminal nodes. For group A, the only event time is 8, with one patient at risk and one death, so

H^A(8)=11=1.0,S^A(8)=e−1=0.368.\hat H_A(8) = \frac{1}{1} = 1.0, \qquad \hat S_A(8) = e^{-1} = 0.368.

For group B, the event times are 1, 2, 3 and 4, with risk sets of 4, 3, 2 and 1, so the increments of the cumulative hazard are the reciprocals of the risk sets: 1/41/4, 1/31/3, 1/21/2 and 1/11/1 at times 11, 22, 33 and 44. The cumulative hazard at each event time is the running sum of those increments:

H^B(1)=14=0.25H^B(2)=14+13=0.25+0.3333=0.5833H^B(3)=14+13+12=1.0833H^B(4)=14+13+12+11=2.0833\begin{aligned} \hat H_B(1) &= \frac{1}{4} = 0.25 \\ \hat H_B(2) &= \frac{1}{4} + \frac{1}{3} = 0.25 + 0.3333 = 0.5833 \\ \hat H_B(3) &= \frac{1}{4} + \frac{1}{3} + \frac{1}{2} = 1.0833 \\ \hat H_B(4) &= \frac{1}{4} + \frac{1}{3} + \frac{1}{2} + \frac{1}{1} = 2.0833 \end{aligned}

so the cumulative values at the four event times are 0.250.25, 0.58330.5833, 1.08331.0833 and 2.08332.0833, and the cumulative hazard at t=4t = 4 is 2.08332.0833. The survival function is the exponential of the negative of each cumulative value: e−0.25≈0.779e^{-0.25} \approx 0.779, e−0.5833≈0.558e^{-0.5833} \approx 0.558, e−1.0833≈0.338e^{-1.0833} \approx 0.338 and e−2.0833≈0.125e^{-2.0833} \approx 0.125.

The risk scores are the sums of the cumulative hazard over the shared grid {1,2,3,4,8}\{1, 2, 3, 4, 8\}. Group A has zero everywhere except at 8, where its cumulative hazard is 1.01.0, and is flat afterwards, so on the shared grid {1,2,3,4,8}\{1, 2, 3, 4, 8\} its risk score is τA=0+0+0+0+1.0=1.0\tau_A = 0 + 0 + 0 + 0 + 1.0 = 1.0. Group B has H^B\hat H_B equal to 0.250.25, 0.58330.5833, 1.08331.0833 and 2.08332.0833 at the event times 11, 22, 33 and 44, and is flat afterwards, so on the shared grid {1,2,3,4,8}\{1, 2, 3, 4, 8\} its risk score is τB=1/4+7/12+13/12+25/12+25/12=73/12≈6.0833\tau_B = 1/4 + 7/12 + 13/12 + 25/12 + 25/12 = 73/12 \approx 6.0833. Group A is ranked as the lower risk, which is what we expect.

PythonIn [20]
CollapseExpand
def step_function(times, values, grid):
    """Evaluate a right-continuous step function on an arbitrary grid."""
    if times.size == 0:
        return np.zeros(grid.size)
    position = np.searchsorted(times, grid, side="right") - 1
    return np.where(position >= 0, values[np.clip(position, 0, None)], 0.0)


hand_time = example["time"].to_numpy(dtype=float)
hand_event = example["event"].to_numpy(dtype=bool)
hand_left = (example["group"] == "A").to_numpy()

hand_stat = logrank_statistic(hand_time, hand_event, hand_left)
hand_curves = {}
for name, mask in [("A", hand_left), ("B", ~hand_left)]:
    hand_curves[name] = nelson_aalen(hand_time[mask], hand_event[mask])

grid_hand = np.array([1.0, 2.0, 3.0, 4.0, 8.0])
hand_risk = {
    name: step_function(t_h, h_h, grid_hand).sum()
    for name, (t_h, h_h) in hand_curves.items()
}
ConsoleOut [21]
CollapseExpand
log-rank statistic for the split: -2.0000
group A: event times [8.]
group A: cumulative hazard [1.]
group A: risk score 1.0000
group B: event times [1. 2. 3. 4.]
group B: cumulative hazard [0.25   0.5833 1.0833 2.0833]
group B: risk score 6.0833

The printed values reproduce the hand calculation. The left child (group A) is the lower-risk child, so the sign of the log-rank statistic is negative. If any printed value disagrees with the arithmetic above, the implementation has a bug, and the worked example is what makes that comparison possible. The split criterion and the terminal estimate are built from the same three numbers per event time: the risk-set size, the number of deaths, and the group membership. Those counts determine the two survival calculations once a grouping is fixed. The tree also needs the covariates and candidate thresholds to decide that grouping.

Implementation in scikit-learn and scikit-survival

We now assemble the pieces into working code. The tree is a recursive function whose only job is to choose the best split, recurse, and store a Nelson–Aalen staircase at the leaves. The forest is a bootstrap loop around the tree, plus the out-of-bag bookkeeping. We use NumPy for the mechanism and then compare with the reference implementation in scikit-survival (sksurv), which supplies the reference estimators and metrics used here.

From a log-rank split to a survival tree

A node stores a leaf step function, a split feature and threshold, and two child references. The class below is deliberately thin. The __slots__ attribute avoids the overhead of a dictionary per node, which matters when a forest contains hundreds of thousands of nodes.

PythonIn [22]
CollapseExpand
class SurvivalNode:
    """A node in a survival tree; leaves hold a Nelson-Aalen step function."""

    __slots__ = ("times", "cumhaz", "feature", "threshold", "left", "right")

    def __init__(self):
        self.times = np.empty(0)
        self.cumhaz = np.empty(0)
        self.feature = None
        self.threshold = None
        self.left = None
        self.right = None

    @property
    def is_leaf(self):
        return self.left is None


class SurvivalTree:
    """A single survival tree grown by maximising the absolute log-rank statistic."""

    def __init__(
        self,
        max_depth=7,
        min_leaf=15,
        min_events=8,
        n_cuts=10,
        mtry=None,
        random_state=0,
    ):
        self.max_depth = max_depth
        self.min_leaf = min_leaf
        self.min_events = min_events
        self.n_cuts = n_cuts
        self.mtry = mtry
        self.random_state = random_state

    def fit(self, X, time, event):
        X = np.asarray(X, dtype=float)
        time = np.asarray(time, dtype=float)
        event = np.asarray(event, dtype=bool)
        self.n_features_ = X.shape[1]
        self.mtry_ = self.mtry or max(1, int(np.sqrt(self.n_features_)))
        self.rng_ = np.random.default_rng(self.random_state)
        self.root_ = self._grow(X, time, event, np.arange(X.shape[0]), 0)
        return self

    def _grow(self, X, time, event, idx, depth):
        node = SurvivalNode()
        node.times, node.cumhaz = nelson_aalen(time[idx], event[idx])
        if (
            depth >= self.max_depth
            or idx.size < 2 * self.min_leaf
            or event[idx].sum() < 2 * self.min_events
        ):
            return node
        features = self.rng_.choice(
            self.n_features_, size=self.mtry_, replace=False
        )
        split = best_split(
            X[idx],
            time[idx],
            event[idx],
            features,
            n_cuts=self.n_cuts,
            min_leaf=self.min_leaf,
            min_events=self.min_events,
        )
        if split is None:
            return node
        node.feature, node.threshold, _ = split
        left = X[idx, node.feature] <= node.threshold
        node.left = self._grow(X, time, event, idx[left], depth + 1)
        node.right = self._grow(X, time, event, idx[~left], depth + 1)
        return node

Three direct stopping checks are in play: the depth limit, fewer than twice the minimum leaf size, and fewer than twice the minimum event count. A node also stops when no valid candidate split is found. Each accepted split enforces both child constraints, but the root can still be terminal with fewer events than min_events. In a bootstrap tree, repeated copies of an observation count repeatedly toward these constraints, so the counts are not numbers of distinct original patients. The feature subset is drawn freshly at each split. These controls limit the fitted partition; they do not guarantee low variance or calibrated leaf estimates.

Predictions use the right-continuous step_function from the worked example and a function that walks a tree for every row. The earlier helper returns the value after the jump at an event time, matching the Nelson–Aalen convention.

PythonIn [23]
CollapseExpand
def tree_cumhaz(node, X, grid):
    """Cumulative hazard on a grid for every row of X, from a single tree."""
    out = np.empty((X.shape[0], grid.size))
    for i in range(X.shape[0]):
        current = node
        while not current.is_leaf:
            if X[i, current.feature] <= current.threshold:
                current = current.left
            else:
                current = current.right
        out[i] = step_function(current.times, current.cumhaz, grid)
    return out

The first tree is a useful object to inspect before the aggregation blurs it. The figure below shows the predicted cumulative hazard at a fixed time over a slice of the covariate space, in which the three remaining features are held at zero. The blocks are the terminal nodes.

PythonIn [24]
CollapseExpand
first_tree = SurvivalTree(
    max_depth=4, min_leaf=25, min_events=12, n_cuts=8, mtry=2, random_state=17
).fit(X_train, t_train, e_train)
grid_1 = np.linspace(
    min(np.quantile(X_train[:, 0], 0.01), X_train[:150, 0].min()),
    max(np.quantile(X_train[:, 0], 0.99), X_train[:150, 0].max()),
    160,
)
grid_2 = np.linspace(
    min(np.quantile(X_train[:, 1], 0.01), X_train[:150, 1].min()),
    max(np.quantile(X_train[:, 1], 0.99), X_train[:150, 1].max()),
    160,
)
grid_1_pad = 0.03 * np.ptp(grid_1)
grid_2_pad = 0.03 * np.ptp(grid_2)
grid_1 = np.linspace(grid_1[0] - grid_1_pad, grid_1[-1] + grid_1_pad, 160)
grid_2 = np.linspace(grid_2[0] - grid_2_pad, grid_2[-1] + grid_2_pad, 160)
G1, G2 = np.meshgrid(grid_1, grid_2)
slice_rows = np.zeros((G1.size, 5))
slice_rows[:, 0] = G1.ravel()
slice_rows[:, 1] = G2.ravel()
hazard_slice = tree_cumhaz(first_tree.root_, slice_rows, np.array([2.0]))[:, 0]
PythonIn [25]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, use_book_style

use_book_style()

plt.figure(figsize=(6.0, 4.0))
ax = plt.gca()
mesh = ax.pcolormesh(
    G1, G2, hazard_slice.reshape(G1.shape), shading="nearest", cmap="viridis"
)
ax.scatter(
    X_train[:150, 0],
    X_train[:150, 1],
    s=6,
    color="white",
    alpha=0.5,
    linewidths=0,
)
ax.set_xlabel("$x_1$")
ax.set_ylabel("$x_2$")
ax.set_title("A shallow tree carves the covariate space into rectangles")
plt.colorbar(mesh, ax=ax, label="estimated cumulative hazard at $t = 2$")
ax.set_xlim(grid_1[0], grid_1[-1])
ax.set_ylim(grid_2[0], grid_2[-1])
ax.grid(False)
plt.show()
VisualizationOut [25]
CollapseExpand
A heat map over x1 and x2 with rectangular blocks of different colours, showing the constant cumulative hazard inside each terminal node, with a scatter of training points on top.
Partition produced by a single shallow survival tree. The color is the estimated cumulative hazard at time 2.0, evaluated on a slice of the five-dimensional covariate space in which the last three features are held at zero. Each rectangular region has a constant tree prediction. The dots project the first 150 training observations onto these two features; their remaining covariates need not be zero. The prediction is constant inside each block, showing the axis-aligned thresholds the tree learns.

The blocks are axis-aligned, and the prediction is constant inside each block. That is the structural reason the tree can represent interactions and thresholds without being told about them. This threshold sensitivity is one reason a single tree can have high sampling variance: a small change in the data can move a threshold, and moving a threshold can change a whole block of predictions.

From trees to a forest

The forest is a bootstrap loop, the aggregation, and the out-of-bag machinery. The staged_* methods are there so that we can look at the effect of the number of trees: they return one row per tree, which lets us compute the ensemble and the out-of-bag estimate for any prefix of the forest.

PythonIn [26]
CollapseExpand
class RandomSurvivalForest:
    """Random survival forest with Nelson-Aalen terminal nodes."""

    def __init__(
        self,
        n_trees=100,
        max_depth=7,
        min_leaf=15,
        min_events=8,
        n_cuts=10,
        mtry=None,
        random_state=0,
    ):
        self.n_trees = n_trees
        self.max_depth = max_depth
        self.min_leaf = min_leaf
        self.min_events = min_events
        self.n_cuts = n_cuts
        self.mtry = mtry
        self.random_state = random_state

    def fit(self, X, time, event):
        X = np.asarray(X, dtype=float)
        time = np.asarray(time, dtype=float)
        event = np.asarray(event, dtype=bool)
        n = X.shape[0]
        self.grid_ = np.unique(time[event])
        rng = np.random.default_rng(self.random_state)
        self.trees_ = []
        self.in_bag_ = np.zeros((self.n_trees, n), dtype=bool)
        for b in range(self.n_trees):
            idx = rng.integers(0, n, size=n)
            self.in_bag_[b, idx] = True
            tree = SurvivalTree(
                max_depth=self.max_depth,
                min_leaf=self.min_leaf,
                min_events=self.min_events,
                n_cuts=self.n_cuts,
                mtry=self.mtry,
                random_state=int(rng.integers(2**31)),
            )
            tree.fit(X[idx], time[idx], event[idx])
            self.trees_.append(tree)
        return self

    def predict_cumhaz(self, X):
        X = np.asarray(X, dtype=float)
        total = np.zeros((X.shape[0], self.grid_.size))
        for tree in self.trees_:
            total += tree_cumhaz(tree.root_, X, self.grid_)
        return total / len(self.trees_)

    def predict(self, X):
        """Risk score: the average cumulative hazard summed over the time grid."""
        return self.predict_cumhaz(X).sum(axis=1)

    def predict_survival(self, X, times):
        cumhaz = self.predict_cumhaz(X)
        index = np.searchsorted(self.grid_, times, side="right") - 1
        evaluated = np.zeros((cumhaz.shape[0], np.asarray(times).size))
        observed = index >= 0
        evaluated[:, observed] = cumhaz[:, index[observed]]
        return np.exp(-evaluated)

    def staged_risk_scores(self, X):
        """One row per tree, holding that tree's risk score for every row of X."""
        X = np.asarray(X, dtype=float)
        return np.array(
            [
                tree_cumhaz(tree.root_, X, self.grid_).sum(axis=1)
                for tree in self.trees_
            ]
        )

    def staged_oob_scores(self, X):
        """One row per tree, with NaN wherever the row was in that tree's bootstrap sample."""
        X = np.asarray(X, dtype=float)
        scores = np.full((self.n_trees, X.shape[0]), np.nan)
        for b, tree in enumerate(self.trees_):
            out_of_bag = ~self.in_bag_[b]
            if out_of_bag.any():
                scores[b, out_of_bag] = tree_cumhaz(
                    tree.root_, X[out_of_bag], self.grid_
                ).sum(axis=1)
        return scores

The predict_cumhaz method is the average over trees, and the predict method is its sum over the grid, so the risk score inherits the averaging convention: it is the sum of the averaged cumulative hazards. The survival function is the exponential of the negative averaged cumulative hazard, evaluated at the requested times. The staged_oob_scores method stores a NaN in the in-bag positions, which is the matrix form of the condition that observation ii only receives contributions from trees that did not see it.

Each tree is grown on a bootstrap sample, so its cumulative hazard is defined on its own event times. To combine the trees we evaluate every staircase on the shared grid self.grid_, the distinct event times of the full training sample, before averaging. The step_function used by tree_cumhaz is right-continuous, so the value at a tree's event time is the value just after the jump; between the tree's event times its step function is flat, which is the Nelson–Aalen convention extended off the event times. Averaging these piecewise-constant functions gives the ensemble cumulative hazard, and exponentiating the negative gives the ensemble survival function.

We now fit the forest and compare it with the reference implementation and with Cox regression. We use Harrell's concordance index: the fraction of comparable observed pairs whose risk ordering agrees with event-time ordering, with half credit for tied risks. Its target can depend on censoring, and it is not an unconditional concordance probability without additional assumptions. We implement it explicitly and check it against the reference implementation, because a metric implemented twice is a metric that is less likely to be silently wrong.

PythonIn [27]
CollapseExpand
from sksurv.ensemble import RandomSurvivalForest as SKRSF
from sksurv.linear_model import CoxPHSurvivalAnalysis
from sksurv.metrics import concordance_index_censored, integrated_brier_score
from sksurv.util import Surv


def harrell_c(time, event, risk):
    """Harrell's concordance index: higher risk must mean shorter survival."""
    time = np.asarray(time, dtype=float)
    event = np.asarray(event, dtype=bool)
    risk = np.asarray(risk, dtype=float)
    comparable = event[:, None] & (
        (time[:, None] < time[None, :])
        | ((time[:, None] == time[None, :]) & ~event[None, :])
    )
    n_comparable = int(comparable.sum())
    if n_comparable == 0:
        raise ValueError("No comparable pairs for concordance")
    difference = risk[:, None] - risk[None, :]
    tied = (np.abs(difference) <= 1e-8) & comparable
    ordered = (difference > 1e-8) & comparable
    return float((ordered.sum() + 0.5 * tied.sum()) / n_comparable)
PythonIn [28]
CollapseExpand
forest = RandomSurvivalForest(
    n_trees=60,
    max_depth=6,
    min_leaf=15,
    min_events=8,
    n_cuts=8,
    mtry=2,
    random_state=0,
).fit(X_train, t_train, e_train)
y_train = Surv.from_arrays(event=e_train, time=t_train)
y_test = Surv.from_arrays(event=e_test, time=t_test)

sk_forest = SKRSF(
    n_estimators=60,
    min_samples_split=20,
    min_samples_leaf=10,
    max_features="sqrt",
    random_state=0,
    n_jobs=-1,
).fit(X_train, y_train)
cox = CoxPHSurvivalAnalysis().fit(X_train, y_train)

risk_forest = forest.predict(X_test)
risk_sk = sk_forest.predict(X_test)
risk_cox = cox.predict(X_test)

The next diagnostic varies the number of trees. We plot both OOB and held-out concordance to illustrate their sampling variation; neither curve is required to rise monotonically or to coincide. These curves are not used to tune the settings in this example. Choosing a forest size or other settings from the held-out test curve would consume its independence as a final assessment.

PythonIn [29]
CollapseExpand
oob_scores = forest.staged_oob_scores(X_train)
test_scores = forest.staged_risk_scores(X_test)


def oob_risk(scores, n_trees):
    """Out-of-bag risk score using the first n_trees trees; NaN where no tree is out of bag."""
    sub = scores[:n_trees]
    usable = np.isfinite(sub).any(axis=0)
    risk = np.full(scores.shape[1], np.nan)
    risk[usable] = np.nanmean(sub[:, usable], axis=0)
    return risk


tree_counts = np.arange(2, forest.n_trees + 1)
oob_curve = []
test_curve = []
for b in tree_counts:
    risk_oob = oob_risk(oob_scores, b)
    ok = np.isfinite(risk_oob)
    oob_curve.append(harrell_c(t_train[ok], e_train[ok], risk_oob[ok]))
    test_curve.append(harrell_c(t_test, e_test, test_scores[:b].mean(axis=0)))
oob_curve = np.array(oob_curve)
test_curve = np.array(test_curve)
oracle_c = harrell_c(t_test, e_test, eta_test)
PythonIn [30]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, use_book_style

use_book_style()

plt.figure(figsize=(6.0, 4.0))
ax = plt.gca()
ax.plot(
    tree_counts,
    oob_curve,
    color=PALETTE["primary"],
    linewidth=1.1,
    label="out-of-bag",
)
ax.plot(
    tree_counts,
    test_curve,
    color=PALETTE["amber"],
    linewidth=1.1,
    label="held-out test",
)
ax.axhline(
    oracle_c,
    color=PALETTE["muted"],
    linewidth=1.0,
    linestyle="--",
    label="oracle (true log hazard)",
)
ax.set_xlabel("number of trees")
ax.set_ylabel("concordance index")
ax.set_title("The out-of-bag curve tracks the test curve")
ax.legend(loc="lower right")
polish_axes(ax)
plt.show()
VisualizationOut [30]
CollapseExpand
Two fluctuating concordance curves across forest sizes, with a horizontal true-signal reference above them in this seeded sample.
Concordance index against the number of trees. The out-of-bag estimate is computed from the trees that did not see each observation, and the test estimate is computed on a held-out sample. The dashed line is the concordance index of the true log hazard, a reference based on the known data-generating signal, not a finite-sample upper bound. The finite-sample gap to the oracle combines fitting, approximation, and sampling effects; it does not identify irreducible lost signal.

The two curves fluctuate as trees are added, and their later values summarize this particular fit and sample. A gap to the true-signal reference can reflect finite data, the chosen split grid and leaf constraints, fitting randomness, and sampling variation. It does not show that a forest could never recover the underlying smooth relationship.

The comparison table also reports integrated Brier score. At each time, it compares the predicted survival probability with the survival indicator 1{T~>t}\mathbb{1}\{\tilde T>t\}, estimating squared error with inverse-probability-of-censoring weights. The reference implementation estimates a marginal censoring distribution from training data. Its validity needs independence appropriate to those marginal weights and positive censoring-survival probabilities over the evaluation interval. If censoring depends on covariates, conditional weights may be needed even when event and censoring times are independent given those covariates. Under those conditions the Brier score assesses probability accuracy, including calibration, rather than ranking alone. We select the time grid from training follow-up before scoring test outcomes.

PythonIn [31]
CollapseExpand
score_times = np.quantile(t_train, np.linspace(0.15, 0.85, 25))
curve_forest = forest.predict_survival(X_test, score_times)
curve_sk = np.vstack(
    [fn(score_times) for fn in sk_forest.predict_survival_function(X_test)]
)
curve_cox = np.vstack(
    [fn(score_times) for fn in cox.predict_survival_function(X_test)]
)
curve_oracle = np.exp(
    -0.20 * score_times[None, :] ** 1.5 * np.exp(eta_test)[:, None]
)
Out[32]:
test C-index test integrated Brier
model
oracle (true log hazard) 0.863 0.097
Cox PH, linear 0.722 0.161
random survival forest, ours 0.832 0.119
random survival forest, scikit-survival 0.840 0.111
ConsoleOut [33]
CollapseExpand
our C-index: 0.8324
scikit-survival C-index for the same risk scores: 0.8324
difference: 0.00e+00

The two concordance calculations agree on this continuous-time sample to the reported tolerance. The helper also specifies how event–censor ties, near-tied risks, and no-comparable-pair inputs are handled; agreement on one sample is not exhaustive validation. This is one seeded train/test comparison with fixed settings, not a population ranking. Both forests use 60 trees, but their split searches and leaf constraints differ, and their survival-aggregation conventions differ. The simulated log-hazard predictor is additive in transformed features but nonlinear in the original x1x_1 and x2x_2. A Cox model supplied with the sine and quadratic transformations could represent it; the raw-feature linear Cox model cannot. The oracle is a known-signal reference, not a guaranteed finite-sample winner.

Risk-score profiles show how the fitted responses change along one feature. For each grid value, we replace that feature in every background row and average the predictions. The plots include the known log-hazard component as a reference and standardize each curve separately to reveal turning points. Forest mortality scores and Cox log-risk scores use different scales, so these profiles do not measure comparable effect magnitudes, causal effects, or prediction errors.

PythonIn [34]
CollapseExpand
def partial_dependence(model, background, feature, grid):
    """Average model prediction while a single feature is set to each value of the grid."""
    values = []
    for value in grid:
        rows = background.copy()
        rows[:, feature] = value
        values.append(model.predict(rows).mean())
    return np.array(values)


def zscore(values):
    values = np.asarray(values, dtype=float)
    scale = values.std()
    return (
        (values - values.mean()) / scale if scale > 0 else np.zeros_like(values)
    )


x1_grid = np.linspace(
    np.quantile(X_train[:, 0], 0.02), np.quantile(X_train[:, 0], 0.98), 25
)
x2_grid = np.linspace(
    np.quantile(X_train[:, 1], 0.02), np.quantile(X_train[:, 1], 0.98), 25
)
background = X_train[:60]

curves = {
    "forest": (
        partial_dependence(forest, background, 0, x1_grid),
        partial_dependence(forest, background, 1, x2_grid),
    ),
    "Cox PH": (
        partial_dependence(cox, background, 0, x1_grid),
        partial_dependence(cox, background, 1, x2_grid),
    ),
}
truth = {
    "x1": 1.2 * x1_grid + 1.2 * np.sin(2.0 * x1_grid),
    "x2": 1.2 * (x2_grid**2 - 1.0),
}

pd_plot_grids = (x1_grid, x2_grid)
pd_plot_curves = {
    "truth": (zscore(truth["x1"]), zscore(truth["x2"])),
    **{
        name: tuple(zscore(value) for value in pair)
        for name, pair in curves.items()
    },
}
VisualizationOut [35]
CollapseExpand
Three standardized response profiles along x1: the known log-hazard component, the forest''s mean risk score, and the Cox model''s mean risk score. The forest can bend while the Cox profile is linear.
Risk-score profiles along x1, averaged over the same 60 background rows. Each response is standardized separately to show its turning points, with the known log-hazard component as a reference. These are different response scales, so their heights and slopes do not compare effect magnitudes or prediction errors. The fitted Cox response is linear and need not be flat in a finite sample.
VisualizationOut [36]
CollapseExpand
Three standardized response profiles along x2: the known log-hazard component, the forest''s mean risk score, and the Cox model''s mean risk score. The forest can bend while the Cox profile is linear.
Risk-score profiles along x2, averaged over the same 60 background rows. Each response is standardized separately to show its turning points, with the known log-hazard component as a reference. These are different response scales, so their heights and slopes do not compare effect magnitudes or prediction errors. The fitted Cox response is linear and need not be flat in a finite sample.

The first profile shows bends in the forest response that the Cox model's linear log-risk response cannot express. The second shows a U-shaped forest response along the quadratic feature. The fitted Cox coefficient for that feature is small but nonzero in this finite sample, and separate standardization stretches it into a visible straight line. A standardized profile therefore reveals shape while concealing relative magnitude. Use the held-out concordance and integrated Brier scores for quantitative prediction comparisons.

The final diagnostic is calibration. The forest predicts a survival curve per patient, so we can group patients by their predicted risk and compare the average predicted curve with the Kaplan–Meier curve of the observed data in that group.

This compares each group's mean predicted survival with its Kaplan–Meier estimate. In this simulation censoring is independent of the event time and covariates, which makes Kaplan–Meier an appropriate comparator within the predicted-risk groups. In an application, this requires independent censoring within each group. Agreement of group averages is one aggregate check and does not establish calibration for every covariate profile. Sampling uncertainty also matters, especially late in follow-up.

PythonIn [37]
CollapseExpand
def kaplan_meier(time, event):
    """Kaplan-Meier product-limit estimate: distinct event times and survival values."""
    time = np.asarray(time, dtype=float)
    event = np.asarray(event, dtype=bool)
    n = time.size
    order = np.argsort(time, kind="mergesort")
    t_sorted, e_sorted = time[order], event[order]
    unique_times, first = np.unique(t_sorted, return_index=True)
    at_risk = n - first
    deaths = np.add.reduceat(e_sorted.astype(float), first)
    keep = deaths > 0
    return unique_times[keep], np.cumprod(1.0 - deaths[keep] / at_risk[keep])


cumhaz_test = forest.predict_cumhaz(X_test)
risk_test = cumhaz_test.sum(axis=1)
low_risk, high_risk = (
    risk_test <= np.quantile(risk_test, 0.3),
    risk_test >= np.quantile(risk_test, 0.7),
)
calibration_curves = []
for name, mask in [("lowest 30%", low_risk), ("highest 30%", high_risk)]:
    km_time, km_surv = kaplan_meier(t_test[mask], e_test[mask])
    calibration_curves.append(
        (
            name,
            np.r_[0.0, forest.grid_],
            np.r_[1.0, np.exp(-cumhaz_test[mask]).mean(axis=0)],
            np.r_[0.0, km_time],
            np.r_[1.0, km_surv],
        )
    )
PythonIn [38]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, use_book_style

use_book_style()

plt.figure(figsize=(6.0, 4.8))
ax = plt.gca()
for (name, pred_times, pred_surv, km_time, km_surv), colour in zip(
    calibration_curves, [PALETTE["primary"], PALETTE["coral"]]
):
    ax.step(
        pred_times,
        pred_surv,
        where="post",
        color=colour,
        linewidth=1.1,
        label=f"{name}: forest",
    )
    ax.step(
        km_time,
        km_surv,
        where="post",
        color=colour,
        linewidth=1.1,
        linestyle="--",
        label=f"{name}: Kaplan–Meier",
    )
ax.set_xlabel("time")
ax.set_ylabel("survival probability")
ax.set_xlim(0.0, 6.0)
ax.set_ylim(0.0, 1.02)
ax.set_title("Predicted and observed survival by risk group")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.18), ncol=2)
polish_axes(ax)
plt.show()
VisualizationOut [38]
CollapseExpand
Four declining survival curves. The low-risk mean prediction lies below its Kaplan–Meier curve; high-risk predictions and observations also differ late in follow-up. Solid and dashed lines distinguish predictions and observations.
Mean predicted survival (solid) and Kaplan–Meier estimates (dashed) for the lowest and highest 30% by predicted risk. The low-risk prediction lies below its observed curve through much of follow-up, and late high-risk estimates also differ. These group-level gaps motivate further assessment; sampling uncertainty and within-group censoring assumptions must be considered.

The low-risk group has higher survival than the high-risk group, but the predicted and observed curves do not coincide. The low-risk mean prediction lies below its Kaplan–Meier curve through much of follow-up, and late high-risk estimates also differ. These gaps motivate further calibration assessment; this single grouped plot is not a calibration certificate. Sparse late risk sets add sampling uncertainty, so a gap alone does not identify its cause.

We repeat the comparison with a linear log-hazard predictor. This supplies a case in which the raw-feature Cox model is correctly specified. Its structural restriction can then help estimation, but it does not guarantee a win or tie in every finite sample, metric, or forest configuration. Read the actual result below as another seeded illustration.

PythonIn [39]
CollapseExpand
rng_linear = np.random.default_rng(4242)
Xl_train, tl_train, el_train, etal_train = make_survival_data(
    500, rng_linear, nonlinear=False
)
Xl_test, tl_test, el_test, etal_test = make_survival_data(
    400, rng_linear, nonlinear=False
)
yl_train = Surv.from_arrays(event=el_train, time=tl_train)

forest_linear = RandomSurvivalForest(
    n_trees=60,
    max_depth=6,
    min_leaf=15,
    min_events=8,
    n_cuts=8,
    mtry=2,
    random_state=0,
).fit(Xl_train, tl_train, el_train)
cox_linear = CoxPHSurvivalAnalysis().fit(Xl_train, yl_train)
Out[40]:
test C-index
model
oracle (true log hazard) 0.804
Cox PH, linear 0.799
random survival forest, ours 0.773

On this linear-data draw, compare each fitted model's concordance with the known predictor's value. Together the two examples illustrate why the predictor specification matters. They do not isolate a universal efficiency penalty, establish an ordering for every dataset, or validate the full survival curves from concordance alone.

Key Parameters

The number of trees (n_estimators in scikit-survival, n_trees here) controls averaging and compute cost. Additional trees can reduce randomization variation with diminishing returns. The examples use 60 trees to keep the scratch calculation manageable; that count is not a validated default for another dataset. Assess stabilization with OOB or validation predictions rather than repeatedly selecting from a final test set.

Our min_events constrains event counts in the children of an accepted split, while min_leaf constrains their observation counts. These controls affect the amount of information available to estimate each staircase. A small late risk set can still create a large jump. Scikit-survival's min_samples_leaf and min_samples_split are observation-count constraints and do not translate into an exact event-count minimum. The settings here are illustrative; select them with the event count, follow-up support, and validation behavior in mind.

The number of features considered at each split (max_features in scikit-survival, mtry here) controls the correlation between the trees. "sqrt" is the reference default and "log2" is another option. Their rounded feature counts depend on the number of features; "log2" is not always the smaller subset, and a value equal to the number of features removes feature-subset randomness. Bootstrap randomness remains, and implementations may also randomize ties or feature order.

The depth (max_depth) and the minimum leaf size (min_samples_leaf) control the bias-variance trade-off. Allowing deeper trees enlarges the available partition and can reduce approximation bias while increasing sampling sensitivity, and the leaf-size rule acts as a smoother. An unrestricted tree is rarely what we want, because the terminal nodes end up with too few events.

Key Methods

In scikit-survival, fit(X, y) takes a feature matrix and a structured outcome array whose first field is the boolean event indicator and whose second field is the numeric observed time. Surv.from_arrays(event=..., time=...) constructs that array; keyword arguments make the intended roles explicit. The scratch class instead takes fit(X, time, event) and assumes finite numeric features and valid nonnegative observed times.

predict(X) returns the risk score, which is the sum of the cumulative hazard over the ensemble's time grid. It is a rank-order summary, not a probability and not a hazard ratio. Larger values mean higher risk.

predict_cumulative_hazard_function(X) and predict_survival_function(X) return arrays of step functions, one per row. They are called with a time vector, and they return the value of the function at that time. For a single prediction, predict_survival_function(X)[0](times) is the survival function evaluated at times. Our predict_cumhaz returns an array on grid_, and predict_survival evaluates the negative exponential of that averaged hazard at supplied times. The reference survival method uses averaged Kaplan–Meier estimates, so it need not return the same survival probabilities.

predict is not the same as predict_survival_function evaluated at a single time. The former is a summary of the whole curve, and the latter is the curve. If two patients have the same risk score but different curves, the score cannot distinguish them, and the curves are what the clinician needs.

Practical Implications

A survival forest is a candidate when nonlinear relationships, thresholds, or interactions matter and the data contain enough events and follow-up to estimate local curves. Risk stratification can be a predictive goal in clinical or operational time-to-event problems. Suitability still needs comparison with simpler predictors and validation under the intended censoring and deployment conditions. A forest fitted for prediction does not by itself supply a confidence interval for a causal treatment effect.

Define the time origin, event, and censoring mechanism before fitting, then check event counts and risk-set support. Conditional independent censoring alone does not guarantee independent censoring within every pooled leaf. Scaling is not required for threshold comparisons, but missing covariates need an explicit strategy: this scratch implementation has no missing-value routing or imputation. Original RSF work includes missing-data methods, so that limitation should not be attributed to all survival forests. Values outside the training range follow the learned thresholds to a terminal node; the prediction is piecewise constant rather than an extrapolated covariate effect.

Check leaf-size and event-count settings together, then assess sensitivity to feature subsampling and forest size with OOB or validation predictions. Lowering mtry changes the available splits and may change both tree correlation and prediction quality; it does not guarantee an improvement when features are correlated. For a comparison with Cox regression, use the same training and test observations, disclose supplied transformations and tuning, and report the actual mortality-grid and survival-aggregation conventions. Scores on different grids or from different fitted models have no shared absolute risk unit.

Evaluation, interpretation, and limitations

Evaluation separates discrimination from probability accuracy. Harrell's concordance uses comparable observed pairs and tied-risk handling; it does not test the predicted survival probabilities and can depend on censoring and follow-up. Grouped Kaplan–Meier comparisons are aggregate diagnostics under within-group independent censoring. Integrated Brier score measures squared survival-probability error over a stated interval with censoring weights under its assumptions. Report the interval, censoring estimator, and whether evaluation was held out, OOB, or used during tuning. Different metrics answer different questions, so a good concordance value alone does not establish calibration.

Interpret the forest through its fitted curves and explicitly defined summaries. Permutation importance asks how a chosen predictive score changes after a feature is permuted; correlated predictors and unrealistic permutations can complicate that interpretation. Partial dependence and individual conditional expectation describe fitted responses rather than causal effects. Hazard ratios between profiles can vary with time and need additional definitions for discontinuous estimated hazards; they are not ratios of log hazards. A strictly increasing transformation preserves a mortality score's ordering, but that does not turn the score into a calibrated event probability or imply pointwise ordering of two crossing survival curves.

The scratch split search recomputes a sorted risk-set calculation for each threshold. At a node with mm observations, logrank_statistic sorts in O(mlog⁡m)O(m\log m) time and then performs linear scans. With up to n_cuts thresholds on each of mtry features, this contributes O(mtry n_cuts mlog⁡m)O(\texttt{mtry}\,\texttt{n\_cuts}\,m\log m) per search, in addition to quantile and slicing work. This implementation favors clarity over optimized reuse. OOB score storage uses a tree-by-observation matrix, and the illustrative concordance helper allocates observation-by-observation arrays, so its memory cost is quadratic. Prediction traverses each tree for each row and evaluates the requested grid; large datasets may need batching or a more efficient implementation.

Before relying on a prediction, check whether the censoring assumptions are plausible, whether follow-up supports the requested horizon, and whether leaves retain enough events and at-risk observations. Informative censoring can bias risk-set estimates and may require an explicit adjustment or sensitivity analysis. A flat estimated tail is an absence of later observed increments, not evidence that no later events occur. Also check measurement quality and covariate shift: either a forest or a linear model can fail after a distribution change, and crossing a threshold is one possible source of prediction sensitivity. Interactions can require several successive splits; they need not appear in one split, and a weak marginal signal can make greedy discovery difficult.

Split selection introduces another limitation: a maximum over many candidate log-rank statistics does not have the null distribution of one prespecified comparison. A variable with more candidate cuts can have more opportunities to attain a large value by chance. Random feature subsets and event-count constraints restrict the search but do not provide multiplicity-adjusted inference. Use the statistic to choose a predictive split, and do not report its unadjusted chi-square or normal tail area as a covariate-effect test after selection.

Summary

A random survival forest extends the random forest to right-censored outcomes by replacing the two ingredients that cannot see censoring. The split criterion becomes the log-rank statistic, which compares the observed and expected number of deaths on either side of a candidate threshold at every event time and standardizes by the hypergeometric variance. The leaf prediction becomes the Nelson–Aalen estimator, which accumulates the increments d/Yd/Y over the leaf's own risk sets and is then exponentiated to give a survival function. Bootstrap sampling and random feature subsampling carry over, while out-of-bag prediction and scoring must respect censoring and leakage constraints. The ensemble is the average of the trees' cumulative hazards, exponentiated once at the end.

The reason to use it is flexibility. The model does not require proportional hazards or linear covariate effects, it discovers interactions and thresholds without being told about them, and censored observations contribute to applicable risk sets while they remain under observation in a sampled tree. The reasons to be careful are the price of that flexibility: the predictions are step functions rather than smooth curves, their flat tails supply no learned extrapolation past the last observed event, the risk score is a rank-order summary on an arbitrary scale rather than a hazard ratio, and the method does not solve informative censoring. OOB predictions can support internal assessment under the same censoring and leakage constraints as other evaluation, and partial dependence can describe an average fitted response, but the result is a description of a relationship rather than a formal test of it.

The natural next step is to grow trees sequentially. Gradient boosting adds fitted components to reduce a chosen training objective, using gradients or related working responses. Improvement depends on the loss, step size, and fitted component, and better training loss does not guarantee better held-out prediction. Boosted Trees develops that construction. Survival boosting can use survival-specific objectives such as Cox partial likelihood; it need not reuse the log-rank splitting criterion used here.

Quiz

Check the log-rank calculation, survival aggregation, and out-of-bag predictions.

Join the discussionAsk a question or share what you learned.

Comments

Add to the conversation

A question, a correction, or an example from your own work.

No comments yet. Be the first to share your thoughts!

A different angle
All writing
Machine Learning · Vol. 3

XGBoost: Extreme Gradient Boosting

Covers XGBoost (eXtreme Gradient Boosting), including second-order Taylor expansion, regularization techniques, split gain optimization, ranking loss functions.

78 min read
Machine Learning · Vol. 3

LightGBM: Fast Gradient Boosting

Covers LightGBM gradient boosting framework, including leaf-wise tree growth, histogram-based binning, GOSS sampling, exclusive feature bundling.

52 min read
Machine Learning · Vol. 3

CatBoost: Categorical Boosting

Covers CatBoost (Categorical Boosting), including categorical feature handling, target statistics, symmetric trees, ordered boosting, regularization techniques.

42 min read

About the author

Reference

Citation details

Cite or share this article.

BIBTEXAcademic
@misc{brenndoerfer2026randomsurvival, author = {Michael Brenndoerfer}, title = {Random Survival Forests}, year = {2026}, url = {https://mbrenndoerfer.com/writing/random-survival-forests-from-scratch}, organization = {mbrenndoerfer.com}, note = {Accessed: 2026-10-30} }
APAAcademic
Michael Brenndoerfer (2026). Random Survival Forests. Retrieved from https://mbrenndoerfer.com/writing/random-survival-forests-from-scratch
MLAAcademic
Michael Brenndoerfer. "Random Survival Forests." 2026. Web. October 30, 2026. <https://mbrenndoerfer.com/writing/random-survival-forests-from-scratch>.
CHICAGOAcademic
Michael Brenndoerfer. "Random Survival Forests." Accessed October 30, 2026. https://mbrenndoerfer.com/writing/random-survival-forests-from-scratch.
HARVARDAcademic
Michael Brenndoerfer (2026) 'Random Survival Forests'. Available at: https://mbrenndoerfer.com/writing/random-survival-forests-from-scratch (Accessed: October 30, 2026).
SimpleBasic
Michael Brenndoerfer (2026). Random Survival Forests. https://mbrenndoerfer.com/writing/random-survival-forests-from-scratch
Newsletter

Stay up to date

Get new articles, book updates, and notes on AI and technology.

Unsubscribe at any time. Privacy policy