T5 Task Formatting › Question Answering as Generation
Language AI HandbookAll chapters T5 Task Formatting › Question Answering as GenerationOne section · Sections in this chapter6 / 15 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Question answering fits the text-to-text paradigm naturally, arguably more naturally than any other classical NLP task. The core operation involves reading a question, consulting some source of knowledge (either a provided context or the model's internal parametric memory), and creating an answer as text. This is inherently about transforming one text into another. Given a question and optional context, generate the answer.
Question answering has historically been divided into several subtypes that previously required different architectures. Span extraction QA, as in the SQuAD benchmark, required models that could identify the start and end positions of the answer span in the provided context. This produced architectures with two classification heads over the encoder output: one for the start position, one for the end. Multi-hop QA required architectures that could reason across multiple documents or passages. Knowledge base QA required linking natural language questions to structured database queries. The text-to-text paradigm subsumes all of these under a single formulation.
# Extractive QA: answer is a span from the context
extractive_qa = {
"input": "question: What is the capital of France? context: Paris is the capital and largest city of France. It is located on the Seine River.",
"output": "Paris",
}
# Multi-hop QA: requires reasoning across facts
multihop_qa = {
"input": "question: Who founded the company that makes the iPhone? context: Apple Inc. manufactures the iPhone. Apple was founded by Steve Jobs, Steve Wozniak, and Ronald Wayne.",
"output": "Steve Jobs, Steve Wozniak, and Ronald Wayne",
}Extractive QA Input: question: What is the capital of France? context: Paris is t... Extractive QA Output: Paris Multi-hop QA Input: question: Who founded the company that makes the iPhone? con... Multi-hop QA Output: Steve Jobs, Steve Wozniak, and Ronald Wayne
The extractive example shows a simple fact lookup where the answer appears directly in the context. The multi-hop example requires combining information from multiple sentences, identifying that Apple makes the iPhone, then finding who founded Apple. This chaining of facts is a form of reasoning that the model must perform through its sequence-to-sequence generation process. The T5 encoder must capture both facts from the context and their relationship, and the decoder must synthesize them into a coherent answer.
# Let's see T5 handle a question
qa_input = "question: What is the capital of Germany? context: Berlin is the capital and largest city of Germany. Munich is the third largest city."
inputs = tokenizer(
qa_input, return_tensors="pt", max_length=512, truncation=True
)
outputs = model.generate(**inputs, max_new_tokens=32)
answer = tokenizer.decode(outputs[0], skip_special_tokens=True)Question: What is the capital of Germany? Answer: Berlin
T5 correctly extracts "Berlin" from the provided context. This shows the open-book QA pattern where the model locates and returns the relevant information from the given passage rather than relying on memorized knowledge. The answer is grounded in the context, which reduces the risk of factual hallucination that plagues purely parametric generation.
Open-book vs Closed-book QA
T5's text-to-text format supports a distinction that reveals something important about what large language models learn during pre-training. The format supports both open-book QA, where context is provided, and closed-book QA, where the model must rely entirely on knowledge encoded in its parameters. This distinction matters both for understanding model capabilities and for designing production systems.
# Open-book QA: context provided
open_book = "question: When was Python created? context: Python was created by Guido van Rossum and first released in 1991."
# Closed-book QA: no context, rely on parametric knowledge
closed_book = "question: When was Python created?"
# Both use the same format, but closed-book relies on knowledge stored in model weightsOpen-book QA gives the answer source explicitly. The model's job is comprehension and extraction: finding the relevant information in the provided text. This is a relatively safe form of QA because the answer is grounded in a verifiable source. If the context says "1991," the model should say "1991," and you can check its work.
Closed-book QA tests whether the model memorized relevant facts during pre-training. T5 demonstrated surprisingly strong closed-book QA performance, suggesting that large-scale pre-training on diverse text encodes substantial world knowledge. This finding was influential in showing that large language models do not just learn linguistic patterns. They absorb and can retrieve factual information. A model trained on enough text will have encountered facts about Python's creation date, Nobel Prize winners, historical events, and scientific discoveries, and can reproduce that information on demand.
The closed-book capability is impressive but comes with significant caveats. Parametric knowledge is static: the model cannot update what it learned at training time. If Python released a new major version after T5's training cutoff, closed-book T5 would not know about it. Knowledge is also imprecise: the model might "remember" a fact but confabulate details around it, creating plausible-sounding but incorrect answers. These limitations of parametric memory motivated the development of retrieval-augmented generation approaches, where models are equipped with access to external knowledge sources and use open-book QA formulations to answer questions grounded in retrieved passages. We will explore these approaches in later chapters.
Abstractive vs Extractive Answers
Unlike span-based extractive QA models that can only copy text from the context, T5 can generate abstractive answers by combining information from the source and drawing conclusions that do not appear verbatim in it. This changes what question answering can accomplish. Traditional extractive models are limited to answers that appear verbatim in the source text. T5 can produce answers that the source text implies but does not explicitly state.
# Extractive: answer is verbatim from context
extractive_example = {
"context": "The Eiffel Tower is 330 meters tall.",
"question": "How tall is the Eiffel Tower?",
"extractive_answer": "330 meters tall",
"abstractive_answer": "It stands at 330 meters.",
}
# T5 might paraphrase or synthesize information
synthesis_example = {
"context": "Marie Curie won the Nobel Prize in Physics in 1903. She also won the Nobel Prize in Chemistry in 1911.",
"question": "How many Nobel Prizes did Marie Curie win?",
"answer": "two", # Not a direct span!
}The synthesis example shows this powerfully. The word "two" does not appear anywhere in the context, yet it is the correct answer. The model must count the Nobel Prizes mentioned and express that count as a word. This kind of reasoning and synthesis goes beyond pattern matching. It requires comprehension and inference.
This abstractive capability is powerful but requires careful training data design. If training answers are always extracted verbatim from context, the model will learn to copy rather than synthesize. If training answers are always abstractive paraphrases, the model may introduce unnecessary rephrasing that deviates from the source material. The nature of the training data shapes the model's generation behavior in subtle but important ways. For most production QA systems, a mix of extractive and abstractive training examples produces models that can handle both types of questions gracefully.