T5 Task Formatting › Task Prefixes
Language AI HandbookAll chapters T5 Task Formatting › Task PrefixesOne section · Sections in this chapter3 / 15 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
How does T5 know which task to perform when given an input? The answer is task prefixes, short text strings prepended to the input that signal the desired operation. This mechanism is remarkably simple yet surprisingly powerful, in effect teaching the model to follow instructions embedded in natural language. The prefix acts as a routing instruction, telling the model how to process the input and what kind of output to generate.
Consider these examples:
"translate English to German: The house is wonderful.""summarize: The stock market fell sharply today after...""question: What is the capital of France? context: Paris is the capital..."
In each case, the colon acts as a natural delimiter between the instruction and the content. The model sees the prefix first, uses it to establish the task context, and then processes the remaining text accordingly. When the model sees "translate English to German:", it knows to treat the following text as English source material and produce German output. When it sees "summarize:", it understands that it should produce a condensed version of the following content. This is conceptually similar to how special tokens work in architectures we discussed in earlier chapters, but instead of learned embeddings with special meanings, T5 uses natural language instructions that the model learns to interpret through training.
This approach exploits the model's core competency in understanding language to specify its behavior. Rather than introducing specialized tokens or architectural modifications for each task, T5 uses the same linguistic representations it applies elsewhere. The model learns that certain word patterns at the start of the input correlate with certain expected output patterns, much as it learns any other linguistic regularity. Think of it as teaching the model a meta-skill: the ability to read a natural language description of a task and then perform that task. This meta-skill is exactly what instruction tuning in later models formalizes and scales up.
The key insight is that natural language prefixes work because they are convenient and real. T5 was pre-trained on large amounts of text that includes imperative sentences and task descriptions. The model has already encountered phrases like "translate this sentence" and "summarize the following" in its pre-training data. These phrases carry semantic content the model understands. When a prefix like "translate English to German:" appears, the model activates linguistic representations associated with translation, source and target languages, and cross-lingual equivalence. This semantic activation is what makes natural language prefixes more effective than arbitrary codes or symbols.
import warnings
from transformers import T5ForConditionalGeneration, T5Tokenizer
warnings.filterwarnings("ignore")
# Load T5 model and tokenizer
model_name = "t5-small"
tokenizer = T5Tokenizer.from_pretrained(model_name, legacy=False)
model = T5ForConditionalGeneration.from_pretrained(model_name)# Different tasks with different prefixes
tasks = [
"translate English to German: The weather is nice today.",
"translate English to French: Hello, how are you?",
"summarize: Machine learning is a subset of artificial intelligence that lets computers to learn from data without being explicitly programmed. It has applications in image recognition, natural language processing, and many other fields.",
]# Generate outputs for each task
results = []
for task_input in tasks:
inputs = tokenizer(
task_input, return_tensors="pt", max_length=512, truncation=True
)
outputs = model.generate(**inputs, max_new_tokens=64)
decoded = tokenizer.decode(outputs[0], skip_special_tokens=True)
results.append((task_input[:50] + "...", decoded))Input: translate English to German: The weather is nice t... Output: Das Wetter ist heute schön. ------------------------------------------------------------ Input: translate English to French: Hello, how are you?... Output: Bonjour, comment êtes-vous? ------------------------------------------------------------ Input: summarize: Machine learning is a subset of artific... Output: machine learning is a subset of artificial intelligence that lets computers learn from data without being explicitly programmed. it has applications in image recognition, natural language processing, and many other fields. ------------------------------------------------------------
The model produces task-appropriate outputs based solely on the prefix: German translation for the first, French for the second, and a summary for the third. The prefix creates an implicit conditioning that shapes the entire generation process. Note that T5-small is a relatively small model, so translation quality may be limited compared to larger variants. The key insight is that the same architecture handles fundamentally different tasks through prefix-based routing.
Prefix Design Principles
The choice of prefix matters more than one might initially expect. T5's authors experimented with different prefix styles and found that clarity and consistency are more important than brevity. The prefix is a task specification, not a mere tag, that the model must parse and act upon. Ambiguous or inconsistent prefixes lead to ambiguous or inconsistent outputs.
Effective prefixes share several characteristics. First, they use explicit task naming: words like "translate", "summarize", and "classify" clearly indicate the operation. Second, they include relevant parameters when needed: "English to German" specifies source and target languages, and these language names carry meaning the model understands from its pre-training corpus. Third, they maintain consistent formatting by using the same pattern "task: input" across all instances. Fourth, they read as natural language instructions that a human would understand.
Natural language prefixes work well because T5 has strong priors about how language works. A prefix like "translate English to German:" uses the model's understanding of what translation means, what English and German are, and how to interpret the colon as a delimiter. This is why natural language prefixes often outperform arbitrary codes or symbols. They activate knowledge the model already has. A prefix like "X12: The house is wonderful." carries no semantic content for the model to use. It would need to learn the mapping from scratch, requiring more task-specific training data. In contrast, a human-readable prefix bootstraps the model's existing language understanding.
Here is how different prefix choices affect the same underlying task:
# Same task, different prefix formulations
prefix_variations = [
"cola sentence: The cat sat on the mat.", # T5's actual format for CoLA
"Is this sentence grammatically correct? The cat sat on the mat.",
"grammar check: The cat sat on the mat.",
]The comments highlight an important point. T5 was trained with specific prefixes like "cola sentence:", so using different formulations at inference time may produce unexpected results unless the model has been fine-tuned on the new format.
The specific prefixes used during training become part of the model's learned behavior. T5's original training used prefixes like "cola sentence:" for the CoLA grammaticality task, "sst2 sentence:" for sentiment analysis, and "translate English to German:" for translation. Using different prefixes at inference time may produce unexpected results unless the model has been fine-tuned on the new format. This highlights an important principle: the text-to-text paradigm is flexible but not magical. The model can only reliably perform tasks it has been trained to recognize through their prefixes. For novel tasks or custom prefix formats, fine-tuning on appropriately formatted data is necessary.
There is a deeper point here about the relationship between prefix design and model capability. When you create a new prefix format for a fine-tuning task, you are making a choice about what the model needs to learn. A very explicit prefix like "classify the sentiment of this product review as positive, negative, or neutral:" contains more information than "classify:", but it also means more of the capacity is used on parsing the instruction rather than reasoning about the content. The optimal balance depends on how much data you have for fine-tuning and how diverse the tasks are. For production systems with abundant fine-tuning data, concise and consistent prefixes are usually better.