Temporal State and Memory › Summary

This chapter turned the abstract fact of partial observability into a concrete design discipline. The through-line was a single question, what should the state be when a single observation is not enough, answered first in principle (condition on history, then compress) and then in architecture (gated recurrence, attention, external stores):

• History is the general fallback under partial observability. A practical memory aims to construct a task-sufficient state when one exists, or a useful fixed-budget approximation otherwise. Learned recurrent states can act as filters or belief-like summaries, but they do not automatically represent normalized Bayesian beliefs.
• Recurrence can struggle over long horizons. Long-range derivatives are products of Jacobians and can vanish or explode; the scalar constant-gain example illustrates exponential behavior, while matrix products depend on time-varying directions and spectra. LSTM and GRU updates provide direct gated retention paths. TBPTT limits direct credit-assignment distance, not forward memory as a hard theorem.
• Short-term and long-term memory are different problems. Filtering recent context, integrating slow quantities, and retaining one-time cues have different requirements and different write policies, and good architectures give them separate compartments rather than forcing one vector to do all three jobs.
• Memory can be compressive or addressable. Recurrent summaries trade detail for compactness. Attention and external stores keep representations separately addressable subject to encoding, capacity, eviction, write, and retrieval losses. Compute, averaging, cutoff, similarity, and staleness are characteristic but overlapping risks rather than one exclusive failure per family.
• Aliasing is the diagnostic lens. Identical observations paired with different target conditionals or disjoint optimal actions mark where observation-only models break. The corridor executed per-timestep evaluation, a held-out latent probe, and PCA visualizations. Controlled reactive-policy baselines are a further recommended behavioral ablation; this example did not execute one.

You now have the vocabulary and the failure-mode literacy to read any recurrent world-model architecture critically: ask what its compartments are, what write policy each uses, what horizon its training spanned, and how you would detect an aliasing failure if one occurred. The next chapter asks a complementary question: instead of compressing history into an opaque vector, can we factor the world into objects and relations, so that both memory and dynamics become structured and compositional?