Dropout › Key Parameters
Michael BrenndoerferApril 27, 20251 min read
Language AI HandbookAll chapters Part 10 of 67Neural Network Foundations
Dropout › Key ParametersOne section · Sections in this chapter16 / 19 ↓
Sections in this chapter19
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for dropout regularization are:
- p (dropout rate): Probability of dropping each unit. Typical values: 0.1 to 0.5 for hidden layers. Lower values (0.1) for input layers or transformer blocks. Higher values (0.5) for fully connected classification heads.
- spatial: Whether to use spatial (channel-consistent) dropout. Recommended for sequence data where temporal consistency matters.
- mc_dropout: Whether to keep dropout active at inference time for uncertainty estimation. Requires multiple forward passes and increases inference cost by the number of samples.
- placement in the network: Apply after the activation function in standard dense layers. In transformers, apply after the attention output and after the feed-forward layer before the residual addition.