T5 Task Formatting › The Text-to-Text Paradigm

Traditional NLP systems treat different tasks as fundamentally different problems, each requiring its own architecture, training procedure, and inference logic. A classifier predicts discrete labels using a softmax output layer. A sequence tagger assigns one label per input token, often using a CRF to capture label dependencies. A machine translation system uses an encoder-decoder with cross-attention to align source and target tokens. A summarizer generates text autoregressively conditioned on a long document. Each of these task types has its own form of output, its own loss function, and often its own architecture. The practical consequence was that NLP researchers and engineers maintained separate systems for separate tasks, with each system requiring its own expertise to build and maintain.

T5 rejects this fragmentation entirely by recognizing a simple fact: text is a universal output format. Any output a model might need to produce, whether a single discrete label, a sequence of tags, a numerical score, a translated sentence, or a structured annotation, can be serialized as a string. The output "positive" is text. The output "B-PER I-PER O B-LOC O" is text. The translated sentence "Das Wetter ist schön" is text. If every output can be expressed as text, and every input is already text, then every NLP task becomes a sequence-to-sequence problem: given an input string, generate an output string. A single model with a single architecture, a single loss function, and a single training procedure can handle them all.

To appreciate why this matters, consider the practical situation before unified frameworks like T5. A practitioner building a multi-task NLP system would need a BERT-based classifier with a task-specific classification head for sentiment analysis, a separate BiLSTM-CRF for named entity recognition, a transformer encoder-decoder for machine translation, and yet another model for abstractive summarization. Each model requires its own training pipeline, its own hyperparameter tuning cycle, and its own deployment infrastructure. Maintaining four separate models is four times the engineering work, four times the failure points, and four times the serving cost. The text-to-text paradigm collapses this complexity into a single model that learns to perform all tasks through the same mechanism: autoregressive text generation.

Beyond the practical benefits, this unification reveals something philosophically interesting about the nature of language tasks. Generation is a general form of prediction. When a model generates the word "positive" in response to a movie review, it is making a prediction equivalent to what a classifier makes by selecting class 1. When it generates "B-PER" in response to the name "Marie Curie," it is making a labeling decision equivalent to what a sequence tagger makes token by token. The text-to-text framing is a software engineering convenience with a broader consequence. It suggests that the boundary between "understanding" and "generation" is more fluid than traditional architectures imply. A model that can generate the correct answer to a question shows understanding of the question and the relevant context. Generation becomes the unified test of linguistic competence.

This unification brings several concrete benefits that practitioners can take advantage of immediately:

• Single architecture: The same encoder-decoder model handles all tasks without modification to its structure
• Shared pre-training: Knowledge learned during pre-training on large text corpora transfers to any downstream task, regardless of what that task looks like
• Multitask learning: Multiple tasks can be mixed in a single training batch, letting positive transfer between related skills
• Zero-shot generalization: The model can attempt new tasks if given appropriate formatting, because it has learned to follow natural language instructions embedded in the input

Beyond these practical benefits, this approach suggests that the boundary between "understanding" and "generation" may be more fluid than traditional NLP architectures imply. A model that can generate the correct answer to a question shows understanding of both the question and the relevant context. A model that generates appropriate entity labels has learned to recognize those entities. Generation becomes the unified test of linguistic competence.