System Identification › Limitations, Failure Cases, and Evaluation

No identified model outruns its data. This deserves more than a bullet list, so let us dwell on the deeper issues.

A central failure mode is confident extrapolation into regions the data never visited. On-distribution fit diagnostics need not flag it because they contain no evidence about those regions. For example, a low-order lift-curve fit around cruise can produce a smooth but physically invalid extrapolation at a 60-degree angle of attack. When such a model drives a planner, the planner may select actions that look favorable precisely where the model is least reliable, then collect new data in that poorly supported region. Aggressive optimization can therefore preferentially select favorable model errors. This is not inevitable for every imperfect model and optimizer, but it is a practical reason to regularize optimism, maintain uncertainty over plausible models, keep planners within validated envelopes, and collect data where the current model admits ignorance. Ensemble disagreement is one inexpensive warning signal, although shared data and inductive biases can make the members confidently agree for the same wrong reason.

The evaluation question is likewise subtler than "low prediction loss." Responsible practice reports, at minimum: one-step metrics (did the model class capture local dynamics?); recursive rollout metrics under realistic action distributions (can anything plan with it?); behavior under held-out excitation regimes and held-out initial conditions (is it identifiable-level good, or merely fitted?); closed-loop performance with a controller designed against it (the ultimate, circular-in-a-good-way test); and calibration of predictive uncertainty (does the 95% interval contain truth about 95% of the time?). Reading the list as a sequence of escalating claims clarifies its logic: one-step competence certifies local fidelity; rollout competence certifies compositional fidelity over the tested horizon; held-out excitation certifies that accuracy survived a change of behavioral regime; closed-loop success certifies usefulness under the feedback the model induces; calibration certifies honesty about its own limitations. Each level licenses decisions the previous one does not. Comparison against the irreducible floor from process noise keeps expectations honest. And because high-dimensional perceptual models can look excellent while missing task-critical abstractions entirely, task-grounded evaluations, measuring whether plans formed from the model succeed, belong beside perplexity-style scores, a theme elaborated when we treat evaluation formally in Part XI. The separation between predictive fidelity and decision usefulness reaches its sharpest form here: a video-prediction model may score superbly on pixel likelihood while a planner relying on it misses obstacles, because the lost abstractions never carried much pixel mass.

On impact: system identification has a long industrial record in flight control, process control, automotive powertrains, and spacecraft attitude systems. The methods and validation practices vary by domain, so those examples are evidence of broad use rather than a claim that every deployed controller follows one recipe. Neural and latent approaches expand the observation spaces and parameter counts that identification can address, while inheriting the same need to state the operating regime, excitation coverage, and noise assumptions under which a model was validated.

Three additional caveats round out the picture:

• Closed-loop coupling. Identification under feedback quietly changes what the data can certify, as noted above; whenever you retrofit a model onto logs whose recorded actions came from a human or algorithmic policy reacting to observations, you are doing closed-loop identification whether or not you say so. Driving logs, interaction traces, and other action-annotated behavioral records often have this policy-dependent structure, so the caveat applies more widely than the word "control" suggests.
• Nonstationarity and adaptation. Systems drift, wear, and reconfigure. Online and adaptive identification extends these estimators with forgetting factors and recursive updates, blurring into the continual-learning questions of Part VI. Recursive least squares with forgetting, for instance, runs the same regression incrementally while exponentially discounting old evidence, converting a stationary batch estimator into one that tracks slow drift, at the price of re-admitting the noise sensitivity that averaging used to suppress.
• Partial observability. When sensors see projections of hidden state, identification of reduced-order models risks conflating modeling error with latent-variable mismatch; resolving the two is the business of Part III. A poorly predicting reduced model raises two confounded diagnoses: the reduction kept too few dimensions, or the prediction mechanism is wrong within the kept ones, and distinguishing them requires tools from the representation-learning chapters ahead.