Hierarchical, Symbolic, Language, and Multi-Agent Planning › Limitations and impact

The four interfaces in this chapter restructure different parts of planning. Options reduce high-level decision frequency. Symbols reduce continuous detail in task search. Language changes how goals and constraints are specified. Multi-agent models represent strategic coupling and can increase computational burden. Their common bargain is that useful structure comes with assumptions whose failure must be detected.

Options trade high-level flexibility for temporal reach. Their internal policies remain adaptive, but call-and-return execution postpones option reselection. Boundaries near task-relevant decision points can work well, yet no location type is universally correct. The governing assumptions are that the skill remains competent over its initiation distribution and that its termination or interruption rule returns control soon enough when its purpose changes.

Symbolic abstractions trade resolution and grounding effort for discrete search. A coarse reliable predicate may be preferable to a fine unstable one, but the choice is task dependent. The model assumes that predicate detectors, operator effects, and exogenous-event handling remain valid over execution. A cup falling while no modeled operator records it violates persistence and can invalidate the plan.

Language trades compact specification for grounding ambiguity. New object names, relations, syntax, and unrepresented concepts can break a grounder; the exact failure rate is empirical and domain specific. Evaluation should separate reference resolution, goal extraction, planning, and physical execution rather than report only end-to-end success.

Multi-agent approximations trade model detail for scalability. Exact solution is hard in many large or partially observable formulations, but tractability depends on the game class and representation. An explicit opponent model can overfit to one behavior distribution; an implicit history model can fail after a policy change for the same reason. Evaluation should vary co-player populations and adaptation regimes.

The compounded problem. Because these abstractions stack, their errors also stack. An error at one layer can propagate to later layers, making diagnosis difficult without targeted evaluations. A language instruction that grounds into a symbolic goal whose predicates come from a world model that is itself imperfect produces a plan whose failure is not attributable to any single layer. Diagnostic evaluation therefore becomes essential: when the agent fails, you need to know which layer failed. Was the instruction misunderstood? Was the grounding wrong? Was the world model inaccurate? Was the option library missing a skill? Was the opponent behaving unexpectedly? Evaluation in hierarchical and multi-agent systems is more about attributing failure than measuring success, and this is the theme we will return to in Part XI: Evaluation and Understanding.

The practical consequence of compounding is that evaluation should probe one layer at a time. Test the grounder on instructions whose correct predicate set you know. Test the symbolic planner on predicate sets whose correct plan you know. Test the option library on tasks within its intended scope. Test the opponent model against a range of opponent behaviors, including ones it was not trained on. These component tests make failures easier to diagnose, but they do not replace end-to-end testing of interactions among layers.

Reusable skills and grounded symbols can make some long-horizon tasks substantially more tractable and inspectable. Language can reduce the cost of specifying tasks. Multi-agent models can improve prediction or coordination when their assumptions match the co-players. Reactive policies, receding-horizon control, search, and learned value functions remain viable alternatives or complements.

The next part, beginning with PILCO, PETS, Ensembles, and MBPO, turns to concrete model-based reinforcement-learning lineages. Those methods study probabilistic dynamics, uncertainty-aware control, and synthetic rollouts. Their machinery can later be combined with hierarchical or symbolic interfaces, but it does not automatically instantiate options, language grounding, or recursive multi-agent beliefs.