Predictive and Generative Objectives › Limitations & Impact

The choices we have discussed are the difference between a model that looks good on a training curve and a model that survives being asked to plan. Every family of objectives has failure modes, and the professional skill is knowing which failure mode is most painful for the task at hand.

Reconstruction and likelihood objectives provide dense supervision and a direct probabilistic interpretation when the likelihood is well specified. They reward modeling observations, including predictable detail that may not affect decisions. Posterior collapse is a separate latent-variable failure: a flexible decoder can model observations without using the latent, leaving the posterior close to the prior. KL schedules, free bits, decoder design, and KL balancing can mitigate collapse but do not guarantee a task-sufficient state. More generally, maximizing likelihood is not the same as maximizing decision quality, a point that Stochasticity and Uncertainty makes from the probabilistic side and that will recur in Part XI.

Likelihood scores observations according to the chosen density and reduction. Its effective emphasis depends on parameterized variances, normalization, discretization, and correlations; it does not generically weight every dimension in proportion to empirical variance. A decision objective instead emphasizes distinctions that change action values. Rare but consequential states may contribute little to average log-likelihood while dominating safety or return. Agreement between likelihood and decision quality therefore requires an argument about coverage, calibration, and task sufficiency rather than an assumption.

Contrastive and embedding-target objectives can avoid pixel-fidelity pressure, while raw-target masked methods retain part of it. Their behavior depends on positive and negative construction, mask policy, target updates, and anti-collapse design. These sensitivities differ from those of likelihood training rather than being uniformly worse. Their loss values also lack a universal cross-method scale, so downstream probes and rollout evaluations remain important.

Multi-step and overshooting losses address a specific structural problem: mismatch between teacher-forced training and recursive use. They do not solve it in general. Longer unrolls cost more and can produce noisier gradients. Under a pointwise squared-error objective, multimodal futures can be averaged. Long-horizon consistency targets can also be unreliable while the representation and transition are still changing. Schedules can help in some settings, but add choices that must be validated per environment.

Balancing multiple losses is consequential because only gradients on shared parameters create direct interference. Naive summation uses fixed scalar weights, but a larger loss value does not necessarily produce the larger shared gradient. Uncertainty weighting, GradNorm, PCGrad, stop-gradient boundaries, and KL balancing encode different assumptions and introduce their own controls. Keep the composition simple, log per-term values and shared gradients, and add machinery only for a diagnosed failure.

These objectives matter because they shape whether a learned state supports the rollouts and predictions a planner needs. A strong observation metric does not certify decision usefulness, while a modest generative score need not rule it out. The next chapters in Part VI therefore treat data and evaluation as part of objective design rather than afterthoughts.