T5 Task Formatting › Classification as Generation
Language AI HandbookAll chapters T5 Task Formatting › Classification as GenerationOne section · Sections in this chapter4 / 15 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Converting classification to generation might seem wasteful. Why generate text when you only need a label? This question highlights the efficiency versus flexibility trade-off that defines the text-to-text paradigm. But the benefits of unification generally outweigh the slight overhead, and the approach proves surprisingly powerful in practice.
To understand why, consider what classification entails. A classifier takes an input and selects one of several possible outputs from a fixed set. In traditional systems, this selection happens through a softmax over a fixed set of class vectors. In T5, the selection happens through generation. The "selection" is the model's choice of which label text to produce. The mechanism differs, but the underlying computation is conceptually similar: the model must encode the input, reason about its semantic content, and produce an appropriate response. What changes is not the reasoning required, but the form of the output.
For binary classification, the model simply generates a label token. The label text itself can encode semantic meaning. Using descriptive labels like "positive" and "negative" rather than arbitrary integers like "0" and "1" means the model can use its understanding of what these words mean. A model that understands English knows that "positive" is associated with favorable evaluations and "negative" with unfavorable ones. This semantic grounding potentially allows the model to classify correctly even for inputs unlike anything it saw during fine-tuning.
# Sentiment classification as text generation
sentiment_examples = [
("sst2 sentence: This movie was absolutely fantastic!", "positive"),
("sst2 sentence: What a waste of time and money.", "negative"),
("sst2 sentence: The acting was superb but the plot was confusing.", "?"),
]Input: sst2 sentence: This movie was absolutely fantastic! Expected: positive Input: sst2 sentence: What a waste of time and money. Expected: negative Input: sst2 sentence: The acting was superb but the plot was confusing. Expected: ?
These examples show how sentiment classification maps to text generation. The model learns to produce "positive" or "negative" based on the input text. The third example with "?" as expected output illustrates an ambiguous case where mixed sentiment makes the correct label unclear. Real training data would need a consistent strategy for handling such cases, whether by requiring the model to pick one label, generate a fine-grained response, or output a special token showing ambiguity.
During training, the model learns to generate the appropriate label text. For SST-2 (Stanford Sentiment Treebank), T5 was trained to output "positive" or "negative". For CoLA (grammaticality), it outputs "acceptable" or "unacceptable". The choice of label text is arbitrary in principle: the model could learn to output "1" for positive and "0" for negative. But descriptive labels let the model use its semantic understanding and tend to work better in practice, especially with limited fine-tuning data, because they are semantically coherent rather than arbitrary symbols.
Multi-class and Multi-label Classification
The text-to-text format naturally extends to more complex classification scenarios without requiring any architectural changes. This is where the paradigm's flexibility shines most clearly. A traditional multi-class classifier needs its output layer sized to the number of classes. Adding a new class means modifying the architecture and retraining. In T5, adding a new class just means introducing a new label string in the training data. The model learns to generate the new label through the same training procedure it already uses. No architectural changes are needed.
# Multi-class: News topic classification
multiclass_examples = [
(
"classify topic: Apple announces new iPhone with revolutionary camera.",
"technology",
),
("classify topic: Lakers defeat Warriors in overtime thriller.", "sports"),
(
"classify topic: Federal Reserve raises interest rates by 0.25%.",
"business",
),
]
# Multi-label: Multiple applicable tags
multilabel_examples = [
(
"tags: A new AI chip powers the latest smartphone.",
"technology, electronics",
),
(
"tags: Tech stocks surge after earnings reports.",
"technology, business, finance",
),
]For multi-label classification, the model generates multiple labels separated by a delimiter. This approach is more flexible than traditional multi-hot encoding because the model can generate any number of labels and even combinations of labels it has not explicitly seen during training. The model learns which labels to generate, the delimiter pattern, and the approximate number of labels appropriate for different inputs. A document about an AI startup going public might generate "technology, business, finance," combining labels it has seen in combination before.
The multi-label case also illustrates a subtle but important point about output ordering. Traditional multi-hot vectors have no ordering: the presence or absence of each class is independent. In the generative approach, labels must be listed in some order. The model will learn to follow the ordering pattern in its training data, which could be alphabetical, by salience, or by the order labels appear in the source text. You should choose an ordering convention and apply it consistently in your training data, because inconsistent ordering can degrade performance by requiring the model to learn multiple valid output formats for the same conceptual answer.
Extracting Probabilities
One apparent limitation of generation-based classification is losing access to prediction probabilities. Traditional classifiers output a probability distribution over labels. This supports calibration and thresholding while also providing an estimate of uncertainty. If a traditional classifier is 51% confident about "positive" and 49% about "negative", you know the prediction is uncertain and might want to abstain or route to a human reviewer. With greedy generation, you just get "positive" with no indication of the model's uncertainty.
T5 solves this by giving access to the decoder's output logits. Since generation proceeds token by token, and each token is selected based on a probability distribution over the vocabulary, we can recover probability information by examining these distributions directly. The key insight is that the generated text is only one sample path; the probability distribution behind it contains rich information about the model's confidence and the relative likelihood of alternative outputs.
import torch
def get_label_probability(model, tokenizer, input_text, labels):
"""Get probability of each label for a classification task."""
# Tokenize input
inputs = tokenizer(input_text, return_tensors="pt")
# Get logits for each label
label_probs = {}
for label in labels:
label_ids = tokenizer(label, return_tensors="pt").input_ids
# Forward pass to get logits
with torch.no_grad():
outputs = model(**inputs, decoder_input_ids=label_ids[:, :-1])
logits = outputs.logits
# Calculate probability of generating this label
# (simplified - real implementation handles multiple tokens)
probs = torch.softmax(logits[0, -1, :], dim=-1)
# Handle case where label tokenizes to single token (index 0 after BOS)
target_idx = label_ids[0, 0].item()
label_probs[label] = probs[target_idx].item()
return label_probsThis approach scores how likely the model is to generate each candidate label given the input. By normalizing these scores across the set of valid labels, we recover a probability distribution equivalent to what a traditional classifier would output. The approach requires knowing the set of valid labels in advance, which is always the case for classification tasks. For tasks with open-ended output spaces, like generation or translation, this probability recovery technique does not apply directly, but for classification it works well.
