Hierarchical, Symbolic, Language, and Multi-Agent Planning › Summary

Large planning problems have several distinct sources of complexity. This chapter separated four ways to structure them rather than forcing them into one compression formula.

• Options turn a primitive action into a temporally extended subroutine with a policy, an initiation set, and a termination condition. They compose via the semi-Markov decision process equations, where transitions have variable duration and the Bellman equation must handle the option length by discounting both the intra-option rewards by $\gamma^k$ and the continuation value by $\gamma^\tau$.
• Symbolic abstractions project task-relevant continuous state onto predicates and option effects onto operators. Fully compiled and integrated task-and-motion architectures make different choices about when to query continuous models.
• Language-conditioned planning can map an instruction to a goal, procedure, reward, or constraint. In the fixed-library example it changes the goal and composition, not the available physical controllers.
• Multi-agent planning accounts for joint actions, private information, and changing co-player behavior. Interactive beliefs, bounded reasoning, opponent models, and CTDE are different approximations with different assumptions.

The unifying pattern is structured factorization. Options compress decision time; predicates compress state description; language compresses task specification; multi-agent models structure uncertainty about other policies. Only the first is explicitly designed to change temporal depth. In every case the interface must be tested against the causal and informational structure of the task.

A second unifying pattern is that every interface relies on assumptions. Options rely on skill competence over their initiation distributions. Symbols rely on grounded predicates and modeled effects. Language relies on semantic alignment between instructions and the planner's representation. Multi-agent methods rely on assumptions about co-player behavior or the robustness of a history-dependent model. Naming those assumptions makes the corresponding evaluation target explicit.

The open design question is how much structure to specify and how much to learn. The decision-centric lineages in the next part provide concrete dynamics and control mechanisms against which that choice can be evaluated.