Time-to-Event Data, Censoring, and Hazard Functions › Visualizing Survival, Censoring, and Hazards

Michael BrenndoerferOctober 25, 20267 min read
Machine Learning from ScratchAll chapters
Part 6 of 15Survival Analysis
Chapter 48Time-to-Event Data, Censoring, and Hazard Functions › Visualizing Survival, Censoring, and Hazards
Reading section 5 of 10 · Visualizing Survival, Censoring, and Hazards · 7 minSection 5 of 10 · 7 min

Section 5 of 10 · 7 min read

Time-to-Event Data, Censoring, and Hazard Functions › Visualizing Survival, Censoring, and HazardsSection view · Sections in this chapter5 / 10 ↓
Reading tools Definitions, links & notes

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

Notes & comments

Before fitting anything, it is worth looking at the shapes involved. The three quantities S(t)S(t), h(t)h(t), and H(t)H(t) move together, and the way they move is easiest to grasp from a figure. We will use a Weibull family because it conveniently covers the three qualitative hazard shapes: constant (exponential as a special case), increasing, and decreasing.

The Weibull survival function with shape k>0k > 0 and scale λ>0\lambda > 0 is S(t)=exp⁡ ⁣(−(t/λ)k)S(t) = \exp\!\left(-(t/\lambda)^k\right), and the corresponding hazard is h(t)=kλ(tλ)k−1h(t) = \frac{k}{\lambda}\Big(\frac{t}{\lambda}\Big)^{k-1}. Here kk controls the shape of the hazard and λ\lambda is the scale parameter measured in the same time units as tt. Note that this scale λ\lambda plays a different role from the exponential rate λ\lambda used in the worked example below, where λ\lambda is the constant hazard. When k=1k = 1 the hazard is constant at 1/λ1/\lambda. When k>1k > 1 the hazard increases with time (wear-out). When k<1k < 1 the hazard decreases with time (infant mortality). Increasing, decreasing, and approximately constant hazards over particular time ranges have been observed in component reliability data. Understanding these survival curves is a first diagnostic tool.

PythonIn [3]
CollapseExpand
def weibull_survival(t, shape, scale):
    return np.exp(-((t / scale) ** shape))


def weibull_hazard(t, shape, scale):
    return (shape / scale) * (t / scale) ** (shape - 1.0)


t_grid = np.linspace(0.01, 20.0, 400)
t_grid_survival = np.linspace(0.0, 20.0, 400)

scenarios = [
    dict(name="Exponential (shape = 1)", shape=1.0, scale=5.0),
    dict(name="Weibull increasing (shape = 2)", shape=2.0, scale=5.0),
    dict(name="Weibull decreasing (shape = 0.5)", shape=0.5, scale=5.0),
]

for s in scenarios:
    s["S"] = weibull_survival(t_grid_survival, s["shape"], s["scale"])
    s["h"] = weibull_hazard(t_grid, s["shape"], s["scale"])

The next two figure cells are split purely for presentation. Both read the precomputed grids and the scenario dictionaries, and neither recomputes the data. The first shows the survival functions. The second shows the hazards.

PythonIn [4]
CollapseExpand
from book_plot_style import PALETTE, polish_axes, use_book_style


def pal(i: int):
    keys = ("primary", "amber", "teal", "rose")
    return PALETTE[keys[i % len(keys)]]


use_book_style()
fig, ax = plt.subplots(figsize=(6.0, 4.0))
for i, s in enumerate(scenarios):
    ax.plot(
        t_grid_survival,
        s["S"],
        color=pal(i),
        linewidth=1.1,
        linestyle=("-", "--", "-.")[i],
        label=s["name"],
    )
ax.set_xlabel("Time t")
ax.set_ylabel("Survival probability S(t)")
ax.set_title("Weibull survival at three shapes")
ax.set_ylim(0, 1.02)
ax.legend(loc="upper right")
polish_axes(ax)
plt.tight_layout()
plt.show()
VisualizationOut [4]
CollapseExpand
Three decreasing curves start at one and cross at time 5. The shape-0.5 curve is steepest near zero and lies highest after the crossing; the shape-2 curve lies highest before the crossing and lowest afterward.
Survival functions for three Weibull shapes with scale 5. All three curves meet at time 5 with survival exp(-1). The shape-0.5 curve falls steepest near zero and has the highest survival after time 5, while the shape-2 curve has the highest survival before time 5 and the lowest after time 5.

The survival plot makes two things visible. First, all three start at S(0)=1S(0) = 1 and eventually decay to zero, as required. Second, the shape parameter changes the shape of the decay. The decreasing-hazard case (infant mortality) drops steeply early and then flattens, so its survival curve has a long tail. The increasing-hazard case (wear-out) stays near one for a while and then falls rapidly. These are two ends of the stated monotone-hazard spectrum. Real data require their own hazard diagnostics.

PythonIn [5]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, use_book_style

use_book_style()
fig, ax = plt.subplots(figsize=(6.0, 4.0))
for i, s in enumerate(scenarios):
    ax.plot(
        t_grid,
        s["h"],
        color=pal(i),
        linewidth=1.1,
        linestyle=("-", "--", "-.")[i],
        label=s["name"],
    )
ax.set_xlabel("Time t")
ax.set_ylabel("Hazard h(t)")
ax.set_title("Weibull hazards at three shapes")
ax.set_ylim(0, 2.4)
ax.legend(loc="upper right")
polish_axes(ax)
plt.tight_layout()
plt.show()
VisualizationOut [5]
CollapseExpand
One flat horizontal line, one line rising steadily from near zero, and one line dropping sharply from a high value and flattening near zero.
Hazard functions for the same three Weibull scenarios, with constant exponential, rising shape-2, and falling shape-0.5 hazards. The constant line reflects a memoryless exponential lifetime, the rising shape-2 hazard describes wear-out, and the falling shape-0.5 hazard describes infant mortality followed by lower risk among survivors.

A constant hazard means the same instantaneous failure rate at each age; for an exponential lifetime it also gives a memoryless residual lifetime. A rising hazard can describe wear-out, while a falling hazard can describe early failures followed by lower risk among survivors. A decreasing population hazard alone does not prove that individual devices become stronger. The hazard grid starts at 0.010.01 because the shape-0.50.5 hazard diverges as t↓0t\downarrow0; the survival grid includes zero.

Cumulative hazard H(t)H(t) is dimensionless and increases at rate h(t)h(t). At a given time, a larger H(t)H(t) corresponds to a smaller survival probability through S(t)=exp⁡(−H(t))S(t)=\exp(-H(t)). The absolute rate of decline of survival is h(t)S(t)h(t)S(t), so a larger hazard alone does not imply a steeper survival curve when the survival levels differ.

PythonIn [6]
CollapseExpand
import numpy as np

cumulative_hazards = []
for scenario in scenarios:
    cumulative_hazards.append(
        dict(name=scenario["name"], H=-np.log(scenario["S"]))
    )
VisualizationOut [7]
CollapseExpand
Three rising cumulative-hazard curves; one rises fastest early, one fastest late.
Cumulative hazard for the three Weibull scenarios. The shape-0.5 curve grows steeply near zero and then flattens, while the shape-2 curve steepens with time. At each time, the scenario with larger cumulative hazard has smaller survival probability because S(t) = exp(-H(t)).

For the censoring plot, imagine devices installed at independent, uniformly distributed calendar dates during a fifteen-unit recruitment period, with follow-up ending at a common administrative cutoff. Each device is followed from installation, so its potential follow-up duration CC is uniform from zero to fifteen and independent of its simulated lifetime TT. This is staggered recruitment without left truncation. Horizontal segments end at Y=min⁡(T,C)Y=\min(T,C); diamonds mark failures and squares mark censorings. A censoring marker locates the end of observation, not an unobserved failure.

PythonIn [8]
CollapseExpand
rng = np.random.default_rng(0)
n_swim = 40
T_swim = rng.exponential(scale=5.0, size=n_swim)
C_swim = rng.uniform(0, 15.0, size=n_swim)
Y_swim = np.minimum(T_swim, C_swim)
delta_swim = (T_swim <= C_swim).astype(int)

order = np.argsort(Y_swim)
Y_swim = Y_swim[order]
delta_swim = delta_swim[order]
PythonIn [9]
CollapseExpand
import matplotlib.pyplot as plt
from book_plot_style import PALETTE, polish_axes, use_book_style

use_book_style()
fig, ax = plt.subplots(figsize=(6.0, 4.5))
for i, (y, d) in enumerate(zip(Y_swim, delta_swim)):
    color = pal(0) if d == 1 else pal(1)
    ax.plot([0.0, y], [i, i], color=color, linewidth=1.1)
    marker = "D" if d == 1 else "s"
    ax.plot(y, i, marker=marker, color=color, markersize=5)
ax.set_xlabel("Observed time Y (simulated units)")
ax.set_ylabel("Subject index (sorted by Y)")
ax.set_title("Observed failures and censoring")
ax.legend(
    handles=[
        plt.Line2D(
            [],
            [],
            color=pal(0),
            marker="D",
            linestyle="-",
            label="Observed event",
        ),
        plt.Line2D(
            [],
            [],
            color=pal(1),
            marker="s",
            linestyle="-",
            label="Right-censored",
        ),
    ],
    loc="lower right",
)
polish_axes(ax)
plt.tight_layout()
plt.show()
VisualizationOut [9]
CollapseExpand
Forty horizontal segments starting at zero, sorted by length. Some end with diamond markers, others with square markers, indicating events and censoring.
Right censoring in a simulated device study, with each horizontal line representing one device ordered by observed time. Diamonds mark observed failures and squares mark censored records, so for censored records the true failure time lies somewhere to the right of the marker.

This is the visual form of the (Yi,Δi)(Y_i, \Delta_i) representation. Read each line as "device ii was observed from 0 to YiY_i; the marker tells us whether the device failed or was still running." The censored lines are not wrong or missing. They carry important information, namely that the true failure time is greater than the endpoint we see. A correct survival likelihood gives those records weight S(Yi)S(Y_i) for the region to the right of the marker, which is exactly the probability that the event had not yet occurred at the moment we stopped observing.

Comments

Add to the conversation

A question, a correction, or an example from your own work.

No comments yet. Be the first to share your thoughts!

Newsletter

Stay up to date

Get new articles, book updates, and notes on AI and technology.

Unsubscribe at any time. Privacy policy