Control as Inference and Active Inference › Worked Example: A Route Posterior in a Grid World

Enough abstraction. Let us compute a control problem entirely by inference and inspect what the machinery actually produces, so that every symbol from the derivations acquires a concrete address on a map.

Consider a $5 \times 5$ warehouse. Our courier robot starts at the bottom-left tile; the charging dock sits at the top-right. Four moves exist: up, right, down, left. Motors are imperfect: with probability $0.15$ the commanded direction is replaced by a direction sampled uniformly from all four moves. The replacement can therefore coincide with the original command; at walls, outward directions are clamped to a no-motion outcome. Each occupied moment costs $0.2$ (time is money), docking grants $10$, and the discounted maximum-entropy solver below uses $\gamma=0.96$. This is the Bellman surrogate described above, not the undiscounted finite-horizon posterior identity. An exact discounted trajectory tilt would instead use time-indexed evidence factors $\exp(\gamma^{t-1}r_t/\beta)$. Boundary clamping makes a wall-adjacent position potentially more robust because an outward replacement becomes a no-motion outcome rather than a lateral displacement, but the computed policy must determine whether that benefit outweighs the remaining chance of losing progress. The mirror-symmetric geometry makes the upward-first and rightward-first route families equivalent.

The inference view tells a richer story. We declare that the robot's actions were generated by some near-optimal strategy, express that declaration as optimality evidence $p(\mathcal{O}_t{=}1 \mid s, a) \propto \exp(r/\beta)$, and compute the posterior over trajectories. What should we expect? The answer depends entirely on where we set the dial, and the three regimes are worth naming before we compute:

• With small $\beta$, optimality evidence is draconian. Only near-perfect routes survive conditioning, and the trajectory posterior concentrates on the family of shortest routes. In this symmetric grid, the eight moves contain four upward and four rightward commands, giving $\binom{8}{4}=70$ monotone routes. Under the implemented replacement model, each commanded step reaches its intended neighbor with probability $0.85 + 0.15/4 = 0.8875$, so every specified monotone state sequence has the same dynamics-prior weight, $0.8875^8 \approx 0.385$. By contrast, $0.85^8 \approx 0.272$ is the probability that no replacement event occurs at all. Wall adjacency matters to robustness under alternative slip outcomes, not to the relative weight of those already-specified shortest trajectories. The exponential tilt suppresses routes that forfeit even a few points of return, so effectively only the winners remain visible.
• With large $\beta$, evidence barely constrains behavior. The posterior spreads diffusely over the whole grid: wanderings, detours, loops, all faintly tinged with the hint that docking is nice. Nothing is forbidden; trajectories are merely graded, gently.
• Between the extremes lies the interesting regime: a corridor of plausibility around the direct route, wide enough to admit multiple near-equivalent paths. Here the posterior encodes something no argmax policy can represent: both the best plan and a calibrated census of all plans that were close contenders.

Inference thereby replaces a single prescribed route with a calibrated probability distribution over routes, and the temperature dial $\beta$ dictates its sharpness. The implementation section turns this narrative into running code and measures the tradeoff the dial governs: as $\beta$ rises, behavior becomes more exploratory and the expected return decays along a measurable curve.

The same example lets us preview active inference's distinguishing trait. Suppose the warehouse contains a door whose opening mechanism the robot's model has never verified. At sufficiently low temperature, a reward-only control-as-inference courier strongly suppresses the costly detour and heads toward the dock. At higher temperature it may sample the door incidentally, but its exploration remains undirected because no information-seeking term values the uncertainty reduction itself. An expected-free-energy courier assigns salience to the mystery. Even at a slight cost in immediate preference satisfaction, peeking at the mechanism can slash its uncertainty and change everything about later decisions, especially if it turns out the door is a shortcut. The second half of the implementation constructs the smallest possible problem exhibiting exactly this contrast and quantifies the trade.