Sim-to-Real Deployment and Continual Correction › Online Monitoring: Innovation and Residual Signals
World Models HandbookAll chapters Sim-to-Real Deployment and Continual Correction › Online Monitoring: Innovation and Residual SignalsOne section · Sections in this chapter7 / 13 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Monitoring compares predictions with observations to identify departures worth investigating. In filtering, an innovation is a new measurement minus its predicted measurement, conditioned on prior information. Our fully observed proxy uses a next-state residual instead. These are related prediction-error signals, but the innovation's noise law depends on the filtering model; it is not interchangeable with every learned dynamics residual.
The residual is defined as the difference between the observed next state and the model's prediction:
where:
- : the observed next state delivered by the sensors
- : the model's prediction of the next state from the current observation and the issued action
- : the residual, a vector of the same shape as the state
- the monitor's scalar magnitude: the norm after dividing the position and velocity residuals by the fixed 1 m and 1 m/s reporting scales, respectively
Assess residuals against the model's specified observation and disturbance law. Zero mean and whiteness are properties to justify under that law, not universal requirements. A mean shift, changing variance, spikes or temporal correlation can motivate tests of dynamics, sensors, timing or operating conditions. Such patterns can have multiple causes: zero-mean parameter errors and constant structural errors are possible, as the earlier taxonomy showed.
Sequential change detection accumulates evidence over time. A likelihood-based CUSUM uses log-likelihood increments for specified null and alternative models, resetting its running statistic when accumulated evidence becomes unfavorable. Centered tabular versions can be derived for particular distributions, such as a Gaussian mean change; an arbitrary centered residual sum need not be a likelihood ratio. A calibrated threshold can support a stated false-alarm property under its assumptions. Accumulation can improve detection of small persistent shifts, but does not make every shift distinguishable or remove the false-alarm/detection-delay tradeoff. The NIST CUSUM handbook describes the centered cumulative and tabular forms. The proxy below uses a simpler consecutive-exceedance rule, not CUSUM.
Choosing thresholds on separate reference data
The practical decision is where to set the threshold. There are two honest options, and they are often conflated.
- A calibrated sequential guarantee. For example, set a threshold to meet a target expected stopping time under a specified null law, including temporal dependence. The claim is conditional on that law or on the validity of a stated robust null class; it must be checked for the deployment process.
- An empirical threshold from reference data. A sample quantile defines a cutoff. Estimate its same-condition exceedance rate on separate held-out data and account for sampling uncertainty and serial dependence. The quantile label alone supplies neither an exact population tail probability nor a sequential false-alarm guarantee.
We will use the second, and be explicit that it is not the first. The distinction matters because a quantile threshold carries no guarantee once the operating conditions drift away from the reference period: if the reference data was collected with a warm motor and the deployment runs cold, the "99th percentile" was the 99th percentile of a different distribution. Stating which kind of threshold you used is not bookkeeping; it determines what you are entitled to claim when the monitor stays quiet.
mon_rng = np.random.default_rng(4242)
def residual_series(predict_batch, states, u_seq):
pred = predict_batch(states[:-1], np.asarray(u_seq, dtype=float))
return normalized_state_error(pred, states[1:])
def collect_residuals(plant, n_eps):
"""Dimensionless residual sequence under the nominal monitor."""
out = []
for _ in range(n_eps):
u = exploration_actions(mon_rng, 120)
s0 = mon_rng.normal(scale=0.4, size=2)
states, _ = rollout(plant, u, s0=s0)
out.append(residual_series(nominal_predict_batch, states, u))
return np.concatenate(out)
# Reference period: the plant the monitor was tuned on.
ref_res = collect_residuals(NOMINAL, 20)
# Held-out period: estimate individual-sample exceedances, not alarm events.
hold_res = collect_residuals(NOMINAL, 20)
# A plant outside the randomized support: the monitor should react.
shift_res = collect_residuals(SHIFT, 20)
THRESH = float(np.quantile(ref_res, 0.99))
false_alarm_rate = float(np.mean(hold_res > THRESH))
# Beginning of the first three-exceedance run; declaration occurs two samples later.
exceed = shift_res > THRESH
detection_delay = None
run = 0
for i, e in enumerate(exceed):
run = run + 1 if e else 0
if run >= 3:
detection_delay = i - 2
breakThreshold (normalized error, reference 99th percentile) : 0.00000 Reference residual median : 0.00000 Held-out same-plant sample exceedance rate: 0.0000 Shifted-plant exceedance rate : 1.0000 Start index of first three-exceedance run : 0 First alarm declaration residual index : 2 This is a held-out single-sample exceedance fraction, not an alarm-event rate. It supplies no sequential or distribution-shift guarantee.
In this noiseless setup, the reference plant is exactly the nominal predictor. Its reference and independent same-plant residuals are zero; with the strict > rule, the held-out sample exceedance fraction is zero, not one percent. That fraction is distinct from the alarm-event rate of the three-consecutive rule. Exchangeability alone does not make a sample 99th-percentile threshold yield exactly one percent exceedances, especially with ties. Even a continuous nondegenerate distribution leaves finite-sample quantile uncertainty. A changed exceedance rate can motivate checks, but is not by itself a diagnosis of drift. Residuals are concatenated from separate 120-step episodes; a detector used on independent runs should reset at episode boundaries. The first alarm here lies wholly inside the first episode, so boundary handling does not affect the reported index.
All shifted-plant residuals exceed this degenerate zero threshold in the current runs. Three consecutive exceedances first occur at samples 0, 1, and 2. The reported index is the start of that run, zero; the detector could only declare after observing its third residual, index 2. This is deliberately favorable evidence from an exact noiseless null, not a realistic detection-delay or false-alarm guarantee. On hardware, noise, time alignment, operating changes, and the required alarm-declaration latency need a separate calibration and safety analysis.
What a residual cannot tell you
There is a hard limit on what a residual monitor can conclude, and it is a limit of identifiability, not of engineering. A residual is a scalar (or vector) summary of everything that differs between model and plant. A parametric drift, a sensor bias, an actuator degradation, and a state that has left the region where the model was fitted can all produce the same residual signature. From the residual alone you cannot tell them apart. This is not a defect of the residual; it is a property of any scalar summary that collapses many causes into one number.
Distinguishing causes requires discriminating information, which can come from existing excitation, additional sensors or a deliberately designed test. Active probing is one option, subject to an approved operating envelope, not always a necessity. Compare explicit hypotheses under the same observation and disturbance assumptions. If they induce identical full measurement laws under the chosen experiment, more of that same data cannot distinguish them. Equal predicted means alone are insufficient for this impossibility argument because their variances or other distributional properties may differ.
The shifted plant is outside the declared randomization family, and the monitor fires. That alarm alone does not establish whether parameters, observation processing or an omitted mechanism changed. The generator is known to us, but the detector does not receive that privileged diagnosis. The next section gates residual predictors on another deployment trajectory; it does not implement a fault-hypothesis comparison or diagnose the monitor's shifted plant.