Synthetic Data › Key Parameters
Michael BrenndoerferJanuary 13, 20261 min read
Language AI HandbookAll chapters Part 20 of 67Data Curation
Synthetic Data › Key ParametersOne section · Sections in this chapter8 / 11 ↓
Sections in this chapter11
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for the synthetic data pipeline are:
- Temperature (generation): Controls diversity of teacher outputs. Higher temperatures produce more varied responses but also more errors. Typical range 0.7-1.0.
- Distillation temperature : Controls how much soft knowledge is revealed. is standard softmax; to reveals dark knowledge effectively.
- Alpha : Mixing weight between hard CE loss and soft KL loss. is pure supervised learning; is pure distillation.
- Quality score threshold: Fraction of examples retained after model-based scoring. Typical values 0.5-0.7 depending on teacher quality and task type.
- Seed diversity: Number of distinct topics, domains, and personas in the seed set. More seeds directly increases output diversity.