System Identification › The Setup: Model Classes and What We Assume

Before any estimation, we fix vocabulary. Throughout this chapter, a discrete-time system generates trajectories. Depending on how lucky we are with sensors, we see one of two things:

• Full-state data: at every time step $t$ we observe the state $x_t \in \mathbb{R}^n$ together with the applied action or input $u_t \in \mathbb{R}^m$. Many simulated settings and instrumented physical setups provide this. Full-state data is rare in the wild but pedagogically precious: with it, identifying dynamics becomes ordinary regression, and all the statistical machinery you already know applies directly.
• Input-output data: we observe only measurements $y_t \in \mathbb{R}^p$ produced by an observation map, $y_t = g_\star(x_t, u_t) + v_t$, never the state itself. This is the standard POMDP-flavored situation from Part II, Chapter 1. Nearly everything interesting in practice lives here, because real sensors measure projections of reality, not reality. Your drone's accelerometer measures acceleration, not position and velocity; its gyro measures angular rates, not attitude.

The underlying generative story is a stochastic discrete-time system:

$$x_{t+1} = f_\star(x_t, u_t) + w_t, \qquad y_t = g_\star(x_t, u_t) + v_t,$$

where $f_\star$ and $g_\star$ are the unknown true maps, $w_t$ is process noise (disturbances inside the dynamics), and $v_t$ is measurement noise (sensor corruption). Including $u_t$ in $g_\star$ permits direct feedthrough, such as an accelerometer responding immediately to a motor command; examples without feedthrough simply ignore that argument. This distinction between process noise and measurement noise matters downstream. Process noise means the world does something unpredictable at each step; even the true model cannot forecast those disturbances, so there is an irreducible floor on rollout accuracy no estimator can beat. Measurement noise means our view is blurred but the underlying evolution may be perfectly predictable; a clever estimator can average the blur away. Confusing the two leads to characteristic mistakes: treating sensor jitter as if physics were random wastes modeling effort and inflates uncertainty where none exists, while treating turbulence as sensor error produces overconfident models that fail exactly when conditions get rough.

We will state our standing assumptions explicitly, because every identification guarantee rests on them:

• Markov assumption: the next state depends on the past only through the current state, so $(x_t, u_t)$ suffices to predict $x_{t+1}$. When the assumption fails, because the system has hidden memory, delays, or slowly varying unmodeled parameters, the standard remedy is to augment the state until Markovianity approximately holds, by stacking lagged observations into the effective state. Most autoregressive models are exactly this trick wearing simple clothes.
• Stationarity: the maps $f_\star$, $g_\star$ and noise statistics do not change over the recording window. A drone whose battery drains or whose payload shifts violates this. Battery drain changes the gain from command to thrust; payload shift changes the inertia. Both turn what looks like one long dataset into a sequence of slightly different systems, and any single fitted model will be a compromise nobody asked for.
• Noise conventions: in the clean theory and simulations below, $w_t$ and $v_t$ are zero mean and temporally uncorrelated unless a section says otherwise; Gaussianity is optional except where maximum-likelihood or efficiency statements invoke it explicitly. For open-loop least squares, current innovations must also be independent of, or at least orthogonal to, past regressors and actions. The worked drone example later uses process noise without measurement noise; the ARX discussion then shows why adding sensor noise changes the plain-OLS error structure.
• Causal exogeneity (for many results): the regressors must be uncorrelated with the current disturbance. Closed-loop data can break this condition when feedback carries serially correlated or contemporaneous noise into the input, entangling signal and error. Feedback alone is not enough to imply bias: with white innovations and regressors built only from past variables, the current disturbance can remain orthogonal to the regressors. We return to this distinction in the excitation section.
• Correct-enough model class: the estimator can represent $f_\star$, or at least the projection of $f_\star$ that matters for our use. The qualifier "projection" matters. A model that omits high-frequency flexibility nobody queries might still be perfect for its consumer; a model missing a mode the planner will hit is broken regardless of its training loss.

Two distinctions from Part I, Chapter 3 deserve repeating here because they organize everything that follows. First, a model can score excellently on one-step-ahead prediction and still be useless for closed-loop decision making, where the model's own outputs are fed back recursively. The reason is compounding: at step one the model sees ground truth, but at step two it sees its own imperfect output, at step three a doubly corrupted estimate, and small per-step errors accumulate geometrically in systems with unstable or lightly damped modes. We will measure this gap numerically later in the chapter, and it will recur in essentially every evaluation we perform. Second, identifiability is not the same as accuracy: an unidentified model can accidentally predict well on the data distribution that generated it while answering counterfactual questions (what happens if we act differently?) catastrophically badly. Planners ask exactly those counterfactual questions, so we care about identification depth, not surface fit. A planning algorithm is, structurally, a machine for proposing unusual action sequences; the unusual ones are precisely where a shallow-fit model has had no chance to be tested, and precisely where it is consulted most earnestly.