Predictive and Generative Objectives › Worked Example

To make the multi-step story concrete, let us train two models on the same synthetic controlled system and compare their rollout error as the horizon grows. The system is a two-dimensional linear dynamical system with a mild nonlinearity:

$$x_{t+1} = A x_t + B a_t + \eta(x_t), \qquad \eta(x_t) = 0.05 \begin{bmatrix} \sin(x_{t,1}) \\ \cos(x_{t,2}) \end{bmatrix}.$$

where:

• $x_t\in\mathbb{R}^2$: state at time $t$
• $x_{t,1}$ and $x_{t,2}$: the two state components
• $a_t\in\mathbb{R}$: scalar action at time $t$
• $A$: linear state-transition matrix (defined immediately below)
• $B$: action matrix (defined immediately below)
• $\eta:\mathbb{R}^2\to\mathbb{R}^2$: deterministic nonlinear residual
• $x_{t+1}$: next state
• $t$: time index
• $0.05$: amplitude of the nonlinear residual

With

$$A = \begin{bmatrix} 0.9 & 0.2 \\ -0.1 & 0.85 \end{bmatrix}, \qquad B = \begin{bmatrix} 0.3 \\ 0.5 \end{bmatrix},$$

where:

• $A\in\mathbb{R}^{2\times2}$: linear state-transition matrix
• $B\in\mathbb{R}^{2\times1}$: maps the action into state space

the linear part is stable and cross-couples the two state dimensions, while the residual makes the transition nonlinear. This system is small enough for a tiny MLP, but still lets one-step and recursive objectives produce measurably different rollout behavior.

The eigenvalues of $A$ have magnitude below one, so the linear part is discrete-time stable; this alone does not prove global stability of the nonlinear controlled system for arbitrary actions.

The chosen matrix has eigenvalues inside the unit circle, so its uncontrolled linear component is asymptotically stable. That does not guarantee that either learned model will be contractive, nor does it determine which training objective wins under a finite optimization budget. The nonlinear residual ensures that the MLP is fitting more than a linear map.

Each trajectory is a sequence of states and actions; the model will be trained to predict state transitions.

We fit two models. The first minimizes one-step MSE over all 4,000 training transitions. The second starts from each sampled trajectory's $x_0$, rolls recursively for $K=5$, and minimizes the average of its five per-step MSE values. The model architecture, initial weights, optimizer, learning rate, and number of optimizer updates match. The training information and compute do not: every one-step update uses all 4,000 transitions, while every five-step update samples 32 trajectories and uses only their first five transitions. This is an intentionally compact demonstration of two training procedures, not a controlled estimate of the causal effect of the objective. The coverage mismatch is a plausible reason the five-step model performs worse in this run.

We then evaluate both models by rolling them out for the full horizon $T = 20$ on held-out trajectories and recording mean squared error at each horizon. In this deterministic run, the five-step procedure is about $7.26\times$ worse at horizon 1 and $33.05\times$ worse at horizon 20. That result is evidence about the procedures as implemented, including their unequal training coverage, not evidence that multi-step objectives are generally inferior.

The example does not establish that either objective is generally better. It shows that their rollout behavior can diverge and that training-budget controls are necessary before attributing a result to the objective itself.