Interpretability and World-Model Debugging › Distinguishing Perception, Dynamics, and Planner Failures
World Models HandbookAll chapters Interpretability and World-Model Debugging › Distinguishing Perception, Dynamics, and Planner FailuresOne section · Sections in this chapter6 / 10 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
We now come to the pipeline-level question: given a bad closed-loop behavior, which component is responsible? The tool is controlled component substitution.
Component Substitution
For a controlled toy, define a baseline, inject one disclosed fault, and replace the affected component with the baseline implementation. Restoring baseline verifies that reversal under the tested protocol. Failure to restore baseline may reflect another fault, an incomplete repair, interface incompatibility, or noise; it is not proof of interaction. In an unknown system, a successful replacement can compensate for another component's error, so test alternative explanations before naming a unique cause.
Three interfaces must be specified before you run the experiment:
| Component | Reads | Writes | Oracle / reference substitution |
|---|---|---|---|
| Encoder / state estimator | observation history; privileged simulator state for the oracle | true simulator state | |
| Transition model | , | predicted | exact discrete dynamics |
| Planner | , transition model | action | reference planner with a longer horizon |
A controlled replacement needs compatible shapes, units, time indices, meanings, and information availability in addition to matching array types. A true-state estimator output must use coordinates that the unchanged transition and planner interpret correctly. If an adapter is required, disclose and test it. Record additional information and computation introduced by the replacement.
Oracles: Information and Budget
An oracle substitution may change information or computation as well as prediction accuracy. It need not change both: exact dynamics can use the same inputs and a similar calculation as an approximate model. Disclose what changes in each comparator.
In this noisy position-only toy, the oracle estimator uses privileged simulator position and velocity. An implementable estimator does not receive that exact pair from the sensing channel. Other fully observed deployments may expose the modeled state directly. The oracle comparison measures a configuration-specific cost gap, not an additive estimator contribution or a guaranteed achievable optimum. A rescue motivates improving state information; it does not prove that one feasible estimator can close the gap or that no alternative repair could help.
With candidate count and per-transition work held fixed, a longer-horizon random-shooting planner evaluates more transitions. Longer lookahead is not intrinsically more correct and need not reduce executed cost. Report candidate count, horizon, objective, and model-evaluation count. Matching candidate count isolates neither compute nor candidate-sequence coverage when horizon changes; a fixed-compute comparison answers a different question.
An oracle replacement is a comparator, not an optimality certificate. For cost minimization, a feasible policy's cost bounds optimal cost from above under the same objective and admissible information, even if the policy is not optimal. A privileged-state comparator may not be feasible for the deployment problem. State the quantity being bounded, the feasible policy class, and the proof before calling a result a bound. This laboratory reports matched sample costs, not rigorous bounds on expected deployment performance.
Faults Interact
Fault effects need not be additive. If two isolated faults increase cost by 10 and 15 units relative to the same baseline, their combined increase may differ from 25. These are hypothetical numbers, not a measured frequency claim. A joint experiment is needed to estimate the interaction in the tested system.
Feedback permits interactions: an estimator or transition change can alter candidate rankings, selected actions, later observations, and subsequent predictions. It need not change all of them; some perturbations leave the selected action unchanged. The combined outcome can be additive, subadditive, or superadditive relative to the specified single-fault contrasts. Measure that contrast rather than assume its sign.
Matched initial cases and random streams support paired comparisons and can reduce noise; they are not required for every valid randomized comparison and do not create additivity. Different actions lead to different trajectories. Episode cost and time-indexed state differences are both meaningful counterfactual outcomes, provided the comparison is defined; they are not necessarily same-state local transition errors. The laboratory includes a joint-fault condition to measure its outcome contrast rather than assume a sum.