T5 Task Formatting › Key Parameters
Michael BrenndoerferOctober 15, 20251 min read
Language AI HandbookAll chapters Part 25 of 67Encoder-Decoder Models
T5 Task Formatting › Key ParametersOne section · Sections in this chapter13 / 15 ↓
Sections in this chapter15
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
When deploying T5 for text-to-text tasks, several parameters materially affect both performance and efficiency:
- max_length: Maximum input sequence length for tokenization. Longer inputs are truncated. T5 variants support different maximum lengths (512 for t5-small and t5-base, 1024 for larger variants). Truncating important context can severely degrade performance on tasks like QA where the answer may appear at the end of a long passage.
- max_new_tokens: Maximum number of tokens to generate in the output. Controls output length and inference time. Setting this too small will cause the model to cut off its output mid-generation, while setting it too large wastes computation.
- num_beams: Number of beams for beam search during generation. Higher values explore more candidate sequences but increase computation proportionally. For simple tasks like single-label classification, beam search with num_beams=1 (greedy decoding) is often sufficient. For longer, more complex outputs, beam search with 4-8 beams can noticeably improve quality.
- skip_special_tokens: Whether to remove special tokens like
</s>when decoding generated output to text. This should almost always be True for the final human-readable output, but should be False if you need to inspect the raw token sequence.