Markov and Partially Observable Decision Processes › Key Parameters
Michael BrenndoerferJune 7, 20261 min read
World Models HandbookAll chapters Part 2 of 12Inference, Decisions, and Control
Markov and Partially Observable Decision Processes › Key ParametersOne section · Sections in this chapter9 / 14 ↓
Sections in this chapter14
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for the gridworld planning experiment are:
- gamma: Discount factor, set to 0.9. It controls how far ahead the agent plans and sets the geometric contraction rate of value iteration.
- slip: Slip probability, set to 0.1 per perpendicular direction. Together with the 0.8 success probability it models the slippery floor.
- Reward schedule: Entering the goal earns 20, entering the pit costs 20, and every other step costs 1, encouraging short, safe routes.
- Tolerance: Sweeping stops when the maximum per-sweep value change falls below 1e-12, placing the returned values within roughly 1e-11 of the exact fixed point and making them effectively exact for this example.