BART Pre-training › Key Parameters
Michael BrenndoerferOctober 19, 20251 min read
Language AI HandbookAll chapters Part 25 of 67Encoder-Decoder Models
BART Pre-training › Key ParametersOne section · Sections in this chapter11 / 16 ↓
Sections in this chapter16
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for BART pre-training corruption are:
- mask_ratio: Fraction of tokens to corrupt (default 0.30). Higher values create more challenging reconstruction tasks but may remove too much signal.
- poisson_lambda: Mean of the Poisson distribution for span lengths (default 3.0). Controls average span size; higher values create longer spans.
- mask_token: Special token replacing corrupted spans. Must match the tokenizer's mask token.
- apply_sentence_permutation: Whether to shuffle sentence order before infilling. Enables document-level structure learning.