T5 Task Formatting › Additional Task Formatting Examples
Language AI HandbookAll chapters T5 Task Formatting › Additional Task Formatting ExamplesOne section · Sections in this chapter7 / 15 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The text-to-text paradigm extends far beyond classification and NER; it also handles QA. Its power is its universality. Essentially any NLP task that can be described as "given this text, produce that text" fits naturally into the framework. Let us examine formatting for additional task types, both to show the breadth of the paradigm and to give you concrete patterns for your own tasks.
Semantic Similarity
Sentence similarity tasks ask whether two sentences mean the same thing. This is a basic capability underlying many applications, from duplicate question detection to paraphrase identification to information retrieval. The text-to-text formulation is straightforward: given two sentences as input, output a label or score.
similarity_examples = [
{
"input": "mrpc sentence1: The company posted revenue of $1.2 billion. sentence2: Revenue reached $1.2 billion.",
"output": "equivalent",
},
{
"input": "mrpc sentence1: The stock fell 5%. sentence2: The stock rose 5%.",
"output": "not_equivalent",
},
]For graded similarity on benchmarks like STS-B, the output is a numerical score. This shows how the text-to-text format handles continuous outputs by simply generating the number as text. The model learns to map pairs of sentences to scores in the range 0 to 5, where 0 indicates completely unrelated sentences and 5 indicates semantically equivalent sentences.
graded_similarity = {
"input": "stsb sentence1: A man is playing guitar. sentence2: A person is playing a musical instrument.",
"output": "4.2", # Score from 0-5
}Generating numerical scores as text is a subtle but important capability. The model must distinguish related from unrelated pairs and calibrate its outputs to match the scale of the training labels. This requires understanding that "4.2" is a higher similarity than "2.1," which requires some arithmetic reasoning within a language generation framework. Research has shown that T5 handles this reasonably well, partly because numbers appear extensively in pre-training text and the model learns their relative magnitudes.
Natural Language Inference
NLI determines whether a hypothesis follows logically from a premise. This task tests logical reasoning and semantic understanding, asking the model to evaluate three possible relationships: the premise entails the hypothesis, the hypothesis contradicts the premise, or neither relationship holds (neutral).
nli_examples = [
{
"input": "mnli hypothesis: The animal is sleeping. premise: The cat is curled up on the couch with its eyes closed.",
"output": "entailment",
},
{
"input": "mnli hypothesis: It is raining. premise: People are carrying umbrellas.",
"output": "neutral", # Possible but not certain
},
{
"input": "mnli hypothesis: The restaurant is empty. premise: The restaurant is crowded with diners.",
"output": "contradiction",
},
]NLI is particularly interesting as a text-to-text task because the required reasoning is not about language patterns but about logical relationships between statements. The entailment example requires recognizing that "curled up with eyes closed" implies sleeping. The neutral example requires recognizing that umbrella use is consistent with rain but does not prove it. The contradiction example requires recognizing that "empty" and "crowded" are mutually exclusive. All three require world knowledge and inference, not just pattern matching.
Text Correction
Grammar and spelling correction fit the text-to-text format naturally. The task is to transform erroneous text into corrected text. This is a sequence-to-sequence problem in the most direct sense: the input is a sentence with errors, and the output is the corrected version. This makes it one of the most natural fits for the text-to-text paradigm.
correction_examples = [
{
"input": "correct: Their going to the store tommorrow.",
"output": "They're going to the store tomorrow.",
},
{
"input": "correct: The quick brown fox jump over the lazy dog.",
"output": "The quick brown fox jumps over the lazy dog.",
},
]Text correction shows how the model must simultaneously understand what the intended meaning is (inferring from context what "their" should be) and know the correct form (the contraction "they're"). This requires both semantic understanding and grammatical knowledge. The text-to-text formulation handles this gracefully: the model learns to produce corrected text through the same generation mechanism it uses for everything else.
Summarization with Length Control
T5 can be trained to follow length specifications embedded in the prefix. By incorporating length requirements into the task instruction, the model learns to produce summaries of varying granularity on demand.
# Summarization with different length targets
summarization_variants = [
{
"input": "summarize in one sentence: [long article text]",
"output": "Brief one-line summary.",
},
{
"input": "summarize in 3 sentences: [long article text]",
"output": "First key point. Second important detail. Final takeaway.",
},
{
"input": "summarize for twitter: [long article text]",
"output": "Ultra-brief summary under 280 chars.",
},
]Length-controlled summarization illustrates the generality of the prefix mechanism. The prefix can encode parameters, not merely name the task. It can specify how the task should be performed. The model learns to interpret natural language length specifications and adjust its output accordingly. This is a primitive form of instruction following: the model reads a natural language constraint and tries to satisfy it during generation.
Structured Data Generation
Even structured outputs can be serialized as text. This shows the full power of the text-to-text paradigm. Any output that can be expressed as a string, including JSON objects, SQL queries, and code, becomes a valid generation target.
structured_outputs = [
# JSON-like output
{
"input": "extract info: John Smith is a 35-year-old software engineer at Google.",
"output": '{"name": "John Smith", "age": 35, "occupation": "software engineer", "employer": "Google"}',
},
# SQL generation
{
"input": "translate to SQL: Show me all customers from New York",
"output": "SELECT * FROM customers WHERE city = 'New York'",
},
# Code generation
{
"input": "python function: calculate factorial of n",
"output": "def factorial(n): return 1 if n <= 1 else n * factorial(n-1)",
},
]These structured output examples hint at capabilities that would become central to later models. The ability to generate JSON or SQL from natural language descriptions, as well as executable code, anticipates the code-generating and structured-reasoning capabilities of models like Codex, GPT-4, and Claude. T5 demonstrated these capabilities at modest scale, and the insight that "any structured output is just text" proved enormously influential in how subsequent systems were designed and trained.