Compositionality and Systematic Generalization › Key Parameters
World Models HandbookAll chapters Compositionality and Systematic Generalization › Key ParametersOne section · Sections in this chapter9 / 12 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for the factorized and monolithic comparison are:
- d: Embedding dimension for the mechanism and world embeddings. Both models use 8, but equal embedding width does not equalize total capacity.
- hidden: Width of the hidden layers in each MLP. The monolithic model has one width-64 MLP, while the factorized model has two. Including embeddings, their trainable parameter counts are 5,001 and 9,906 respectively.
- lr: Adam learning rate, set to 0.001 for both models so neither receives a more favorable optimization setting.
- steps: Number of minibatch gradient steps (4000), the full training budget each model receives on the pooled training transitions.
- batch: Minibatch size (512) of transitions drawn at each step from the pooled training data.
The data, optimizer, learning rate, minibatch size, and training steps are matched, but the parameter budget is not. Consequently, this run supports a mechanism demonstration (unseen world embeddings remain untrained while shared mechanism embeddings receive updates), not the causal claim that structure alone produced the full performance difference. A capacity-controlled experiment would add a matched-width factorized model or a larger monolithic control and rerun the comparison under the same seeds.