T5 Architecture › Worked Example: Fine-Tuning T5 for Question Answering

To see the text-to-text framework in action end-to-end, let's trace through what happens when T5 is fine-tuned and then used for a specific task. We will use extractive question answering as our example: given a passage and a question, generate the answer as text.

During fine-tuning, each training example is formatted as follows. The input becomes something like: question: What is the capital of France? context: France is a country in Western Europe. Its capital city is Paris, which is also the country's largest city. The target is: Paris. The model receives this formatted string as its encoder input, and the decoder is trained to generate the target string with teacher forcing.

The model already understands English from pretraining. Fine-tuning teaches it to associate the question: and context: prefix pattern with the specific behavior of extracting or generating a concise answer. After a relatively small number of fine-tuning steps on labeled question-answer pairs, the model becomes proficient at this task. This is the transfer learning payoff: pretraining builds general language understanding, and fine-tuning specializes it.

At inference time, the encoder processes the question-context string and builds a bidirectional representation. The decoder then generates an answer token by token, using cross-attention to reference the encoder's representation of the passage at each step. The first generated token might be a capital letter ("P"), the next the rest of the word ("aris"), and the decoder then produces an end-of-sequence token. The final output, "Paris," is returned as the answer.

The key insight from this example is that T5 does not need to be told explicitly where in the passage the answer is located (as span-extraction models like BERT must do). Instead, it generates the answer text directly, which means it can handle questions whose answers require paraphrasing or synthesis rather than verbatim extraction. This flexibility comes at a cost: the model's output must be post-processed to verify it matches an expected answer, whereas span extraction guarantees the output is a substring of the input. In practice, researchers use both approaches depending on the task requirements.