System Identification › Neural System Identification from Trajectories

Everything so far leaned on structure we rarely enjoy: global linearity, low order, scalar outputs. The interesting worlds are nonlinear, high-dimensional, and partially observed. Neural system identification replaces the hand-designed model class with a learned function, typically producing what modern readers would simply call a world model's dynamics module. This equivalence is worth stating bluntly: if you take the classical identification pipeline, swap the linear predictor for a network, the lagged-history feature map for a learned encoder, and the squared one-step loss for a probabilistic likelihood, you have reinvented the core architecture of contemporary world modeling. The main changes are scale, generality of the observation spaces, and engineering discipline around training. The conceptual issues (excitation, compounding, and noise-model specification) carry over, but their classical remedies require adaptation rather than transferring unchanged.

The dominant architecture keeps a recurrent latent state $z_t$ instead of the true state $x_t$, and learns three maps jointly from raw trajectories. The division of labor mirrors the classical pipeline exactly, which is why classical vocabulary remains useful for diagnosing failures:

• Encoder $q(z_t \mid o_{\le t}, u_{<t})$: compresses the observation-and-action history into a latent summary, playing the role of the belief state from our filtering chapter, but parameterized by a recurrent network or transformer rather than a Bayesian recursion. Where the Bayesian belief tracked an exact posterior distribution, the encoder learns whatever compression suffices to support accurate prediction, which may discard details irrelevant to future outcomes.
• Transition model $\hat z_{t+1} = \hat f_\theta(z_t, u_t)$: the actual dynamics hypothesis, trained to predict its own next latent state (often as a distribution, following the probabilistic styling of our filtering discussion). This component plays precisely the role $f_\star$ played in our setup section; everything the classical theory said about the transition function's responsibilities applies here.
• Decoder $\hat o_t = \hat g_\phi(z_t)$: ties latents back to measurable observations so the whole thing is supervised by prediction of real data. When the predictive likelihood is expressed through this decoder, latents that do not support accurate observational predictions cannot lower the predictive loss, however tidy their internal dynamics. An explicit reconstruction loss strengthens that grounding but is an optional auxiliary term in the objective below.

At the conceptual level, training targets a one-step or few-step predictive log-likelihood over transitions drawn from the data-collecting policy, such as $\min_{\theta,\phi} \sum_t -\log p(o_{t+1} \mid o_{\le t}, u_{\le t})$, optionally with auxiliary terms, reconstruction losses, and regularization. Latent-variable implementations usually optimize a variational bound or sampled approximation rather than this exact marginal likelihood.

where:

• $o_t$: the raw observation received at time $t$, whatever the sensors actually stream
• $o_{\le t}$: the entire observation history up to and including time $t$
• $p(o_{t+1} \mid o_{\le t}, u_{\le t})$: the model's predictive distribution over the next observation given past observations and the action history through the latest command
• $\theta, \phi$: trainable parameters of the encoder and transition model ($\theta$) and of the decoder ($\phi$). Architectural choices (recurrent networks, gated units, attention, state-space layers like S4/Mamba-style selective SSMs) supply inductive biases about memory and timescale; we survey these in Part III, Chapters 3 and 4, and the full lineage (emergent latent dynamics models such as E2C, PlaNet, Dreamer, and successors) belongs to Part V and Part VIII.

Reading the objective against the classical checklist makes several things visible at once. The loss is one-step (at best few-step), so it grades the model exactly where Figure @fig-rollout-compounding warned us grading is cheapest and decisions demand the most. The noise model is implicitly "outputs are contaminated, dynamics may be treated as flexible," inheriting ARX-style assumptions without anyone declaring them. And nothing in the objective penalizes confidently wrong predictions in regions the data-collecting policy never visited, because no data lives there to score. None of these observations amounts to a flaw in the recipe; each flags a place where additional machinery must be bolted on deliberately rather than inherited for free.

Neural identification also admits hybrid forms closer to classical spirit. Physics-informed and grey-box models hard-code trusted structure, estimate unknown physical parameters, and may learn a residual for whatever the trusted equations omit. Classical operator inference fits reduced linear or polynomial operators by least squares on prescribed lifted features; related Koopman-autoencoder variants learn the lifting itself. Implicit training on multistep rollout errors explicitly buys closed-loop quality at gradient cost. The grey-box strategy is especially attractive in engineering contexts where some physics is trustworthy and some is not: encoding conservation laws removes entire families of physically impossible predictions from consideration, shrinking what the data must teach and protecting the parts of the model that matter most for safety.

Four recurring failure surfaces deserve explicit treatment. Three have direct classical analogues, while optimization through long learned recurrences is a modern training issue that interacts with the older questions of excitation and conditioning:

• Compounding rollout error. Multistep autoregressive prediction feeds each guess back in, so small per-step bias can compound rapidly, especially through unstable modes. Mitigations include multistep objectives, scheduled teacher forcing, and per-step spectral or Lipschitz constraints. These constraints bound local amplification, but the realized rollout rate depends on products of Jacobians—the associated Lyapunov exponents or joint spectral behavior—not on the eigenvalues of any single Jacobian alone.
• Distribution shift and model exploitation. Data collected under one policy trains a model accurate near that policy's trajectory tube. A planner then queries the model off-tube, where it is silently wrong, acts foolishly, and steers the next data collection somewhere even stranger. A decision procedure can actively seek out the region where model validity lapses, because optimization concentrates wherever errors happen to favor unrealistically optimistic outcomes. Model-based reinforcement learning (Part II, Chapter 6) turns this cycle into an algorithm; the epistemic lesson belongs in every practitioner's head regardless.
• Similarity-transform ambiguity. Latent coordinates are identified only up to transformations preserving behavior, so latents need not align with interpretable physical quantities. If you followed the linear identifiability discussion, you will recognize this instantly as the structural ambiguity from decades ago, generalized to arbitrary learned encodings, and with the same fundamental status: it reflects what input-output information can determine, not a defect of any particular training run. For planning this is harmless; for scientific modeling it is a reproducibility hazard, since two laboratories fitting the same data can obtain latents differing by an arbitrary reparameterization yet making identical predictions, defeating naive comparisons of learned variables across studies.
• Optimization pathologies through time. Backpropagating through hundreds of recurrence steps meets vanishing or exploding gradients, while rare events contribute little loss mass when the data under-excites them. Classical excitation discipline helps diagnose the coverage problem, but it does not by itself solve long-horizon optimization. A nominally excellent model can therefore remain blind to exactly the transitions a cautious planner most wants predicted accurately.

To make the stakes concrete, we study a forced damped pendulum, smooth and innocent-looking, but governed by $\sin$ of position. The pendulum is a useful adversary for a linear predictor on the native variables $(\theta,\omega,u)$: no affine map on those variables can reproduce the sine term exactly over a nondegenerate interval. A lifted linear model containing $\sin(\theta)$ could represent the Euler step, so the limitation belongs to the chosen features rather than to every conceivable linear representation. We fit a linear model and a small feedforward network on identical lagged feature definitions and rollout protocols, using separate independently generated training and validation trajectories.

First the environment and data. The state is $(\theta, \omega)$: angular position and velocity; the action is applied torque.