Iterative Alignment › Key Parameters
Michael BrenndoerferJanuary 5, 20261 min read
Language AI HandbookAll chapters Part 37 of 67Alignment and RLHF
Iterative Alignment › Key ParametersOne section · Sections in this chapter8 / 12 ↓
Sections in this chapter12
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for the iterative alignment implementation are:
- beta: The temperature parameter controlling the strength of the KL penalty (typically 0.1). Higher values keep the model closer to the reference.
- rolling_reference: Boolean showing whether to update the reference model at each iteration (
True) or keep it fixed (False). - num_iterations: The number of cycles of generation and training to perform.
- num_pairs: The number of response pairs generated per prompt for preference collection in each round.