Whisper Architecture › Key Parameters

The key parameters for Whisper inference control both the model's architecture and its generation behavior:

• model_size: The variant (tiny, base, small, medium, large) determining layers, width, and attention heads. Larger models reach lower word error rates but require more memory and compute.
• encoder_layers / decoder_layers: Number of transformer blocks in each component (6 for base, up to 32 for large). Depth lets hierarchical feature extraction.
• d_model: Hidden dimension (512 for base, 1280 for large). Wider representations can encode more complex acoustic and linguistic patterns simultaneously.
• encoder_attention_heads: Number of attention heads (8 for base, 20 for large). More heads allow the model to attend to more types of acoustic relationships in parallel.
• vocab_size: Total tokens including text, language identifiers, task specifiers, and timestamp tokens. The extended vocabulary is what lets the multitask format.
• return_timestamps: Boolean flag letting temporal alignment prediction at 0.02-second resolution. When enabled, the model interleaves timestamp tokens with text during generation.
• forced_decoder_ids: List of (position, token_id) tuples conditioning generation on specific language or task tokens. Use this to override automatic language detection.
• max_length: Maximum sequence length for generation (default 448 tokens). Setting this too low can truncate long transcripts.