Hierarchical, Hybrid, and Omnimodal Models › Limitations and Impact

Hierarchical, hybrid, and modular architectures add failure modes beyond those of a single-level model. Hierarchy changes credit assignment: it introduces inter-level attribution and backup-design problems, while a well-chosen temporal abstraction can shorten long-horizon credit paths. Sparse slow updates create fewer slow computation nodes, but each node may aggregate gradients from many fast-step losses; signal strength depends on the objective and gradient paths. A fast predictor can also compensate for a drifting slow state unless training or evaluation checks the intended representation. Learned termination and update boundaries may fail only in closed-loop rollouts. These are risks of temporally sparse hierarchies, not mechanisms exercised by the worked example's per-step history encoder. Evaluate them with decision-centric and cross-level metrics in addition to one-step error.

Hybrid models inherit assumptions from their non-learned parts. If a simulator is structurally wrong, an additive residual may still correct it when the mismatch is observable and representable, but the task can become harder than learning a different model or boundary. Spurious predicates and missing axioms can produce invalid plans; execution monitoring, uncertainty, and abstraction repair help limit that risk. Differentiable hybrids add another hazard because a soft relaxation's gradient need not match the hard objective. A useful structural prior can improve efficiency when the learned component can represent the remaining task-relevant mismatch. Residual learning is therefore a conditional design choice for simulator hybrids, not a rule for every hybrid architecture.

Modular and omnimodal architectures face collapse, dominance, and shortcut risks. Aggressive sparse routing can concentrate traffic without an effective balancing design. Training and inference can diverge when they use different noise, temperature, truncation, capacity rules, or input distributions. In multimodal training, the objective may over-rely on whichever stream is easiest, whether vision, audio, touch, or another modality, and a shared latent may encode cross-modal shortcuts. A reconstruction score can therefore coexist with poor control utility. Use decision-aware objectives when task relevance is the problem, and combine them with balancing, masking, missing-input tolerance, uncertainty, or anti-shortcut interventions appropriate to the observed failure.

These architectures have delivered gains under specific conditions. Temporal abstraction can extend effective planning horizons; simulator hybrids can improve real-robot data efficiency when the simulator supplies a useful prior; broad multimodal architectures support training across mixed datasets and adaptation to deployment inputs; sparse MoE layers can add parameter capacity while evaluating few experts per example. None is automatic. Each exchanges a simpler architecture for additional assumptions, diagnostics, and failure modes.