Active Perception, Dual Control, and Exploration › Summary

Active perception and dual control study actions whose physical and informational consequences are optimized jointly. The key takeaways are that information gain quantifies the informational consequence, curiosity motivates pursuing it, and safety constrains how far the pursuit can go.

• Belief-MDPs make knowledge a state. Under a known POMDP and exact filtering, the posterior is a sufficient statistic, so information affects task value through posterior-contingent future actions.
• The dual effect requires control-dependent information. Standard LQG separation is a special case. Nonlinearity, multiplicative control, or parameter uncertainty can reintroduce probing when actions change what later observations can reveal.
• Information gain names its target. For next-state information it equals $H(b_a^-)-\mathbb{E}_oH(\tau)$ and the expected $D_{\mathrm{KL}}(\tau\|b_a^-)$. In Gaussian approximations it uses an expected posterior log determinant; a single log-determinant ratio needs observation-independent posterior covariance.
• Information gain is not value of information. The first is measured in nats; the second is measured in reward. Information can be positive while task value is zero, and no numerical ordering follows without utility-scale assumptions.
• Curiosity is the motivation to pursue information, often implemented as an intrinsic reward on top of extrinsic reward, with families spanning count-based, prediction-error, ensemble-disagreement, information-theoretic, empowerment, and latent-novelty bonuses. Their proxy-dependent failures include noisy-TV attraction, nuisance novelty, miscalibrated disagreement, and exploration collapse.
• Safety criteria are not interchangeable. Expected CMDP budgets, violation regret, chance constraints, invariant sets, CVaR, and shields make different claims over different randomness. Every guarantee is conditional on its method's model, feasibility, confidence, or fallback assumptions.
• The tradeoff is context-dependent. The optimal quantity of exploration changes with the belief, the horizon, the cost of error, and the safety budget. The dual weight and the exploration strategy must therefore be tuned per problem rather than chosen once.

We have now covered the inward-facing half of the planning-and-agency part: how an agent can act on its own uncertainty to gather information, and how safety criteria restrict that choice. Next, in [Hierarchical, Symbolic, Language, and Multi-Agent Planning], we turn outward and consider agents that plan at multiple levels of abstraction, coordinate with other agents, and use language as a planning substrate. The same belief and information primitives reappear there across longer temporal and social structures.