Question Answering › Extractive QA: Finding Answers as Spans

Extractive QA takes a specific form: given a question $q$ and a context passage $c$, find the substring of $c$ that answers $q$. The answer is not generated. It is identified as a contiguous token span $[i, j]$ within $c$, where $i$ is the start index and $j$ is the end index.

This formulation has several advantages. The answer is guaranteed to come from the passage, making hallucination impossible. Evaluation is easy: compare predicted span to gold span. And the task can be framed as a simple classification problem over token positions, which fits naturally into the fine-tuning paradigm for pretrained language models. The model does not need to invent new words; it only needs to recognize which words in the provided passage are the answer.

The constraint that the answer must be a contiguous span from the passage is also a limitation. Many real questions do not have answers that appear verbatim in text. Counting questions ("How many planets are in the solar system?") may require numerical computation. Comparative questions ("Which country has a larger population, India or China?") require comparing values from possibly different parts of the text. Abstractive questions ("What caused World War I?") require synthesizing information into a narrative. Extractive QA handles these poorly or not at all. But for the enormous class of factoid questions ("Who invented the telephone?", "When did the French Revolution begin?", "What is the capital of Australia?"), span extraction works very well.

Given a tokenized passage of length $n$, extractive QA reduces to predicting two integer positions:

$$P_{\text{start}} \in \{1, \ldots, n\}, \quad P_{\text{end}} \in \{P_{\text{start}}, \ldots, n\}$$

where $n$ is the number of tokens in the passage, $P_{\text{start}}$ is the index of the first answer token, and $P_{\text{end}}$ is the index of the last answer token. Together they define a contiguous span within the passage.

A BERT-based model encodes the concatenated question and passage, then applies two learned linear layers over the output representations to compute start and end logit scores for each token position:

$$s_i = \mathbf{w}_s^\top \mathbf{h}_i, \quad e_j = \mathbf{w}_e^\top \mathbf{h}_j$$

where:

• $\mathbf{h}_i \in \mathbb{R}^d$: the hidden state vector at position $i$ from the encoder, with dimension $d$ matching the model's hidden size
• $\mathbf{w}_s, \mathbf{w}_e \in \mathbb{R}^d$: learned weight vectors, one for start positions and one for end positions
• $s_i, e_j \in \mathbb{R}$: scalar logit scores; higher values mean the model believes position $i$ is the start (or $j$ is the end) of the answer

These linear projections are the only new parameters added on top of the pretrained encoder. Everything else (the self-attention layers, feed-forward networks, layer normalization) carries pretrained weights that are fine-tuned with the QA data. The small number of task-specific parameters means the model adapts quickly with relatively little labeled data, often achieving strong performance with just a few thousand training examples.

The scores are then converted to probabilities via softmax over all positions in the passage:

$$P(\text{start} = i) = \frac{\exp(s_i)}{\sum_k \exp(s_k)}$$ $$P(\text{end} = j) = \frac{\exp(e_j)}{\sum_k \exp(e_k)}$$

The denominator sums over all token positions, making sure the probabilities sum to 1 across the passage. The model selects the span $(i, j)$ that maximizes the joint score $P(\text{start} = i) \cdot P(\text{end} = j)$, subject to $i \leq j$ and $j - i < L_{\max}$ (a maximum span length constraint, typically 30-50 tokens, to prevent selecting absurdly long spans).

One important subtlety is that the start and end probabilities are computed independently: the model assigns a probability to each position being the start, and separately assigns a probability to each position being the end. The joint selection enforces the constraint that the end must come after the start, but the model does not directly model the joint distribution $P(\text{start} = i, \text{end} = j)$. This independence assumption is computationally convenient and works well in practice, since the relevant span is usually short and the contextual embeddings carry enough information that start and end predictions are implicitly coordinated.

During training, the model minimizes the sum of cross-entropy losses over start and end positions:

$$\mathcal{L} = -\log P(\text{start} = i^*) - \log P(\text{end} = j^*)$$

where $i^*$ and $j^*$ are the ground-truth start and end positions annotated in the training data. Minimizing this loss pushes the model to assign maximum probability to the correct span boundaries.

The positional relationship between the question and passage matters. Because BERT uses bidirectional attention, every token in the passage attends to every token in the question, and vice versa. This cross-attention between question and passage allows the model to compute question-conditioned representations of each passage token. The hidden state $\mathbf{h}_i$ for a passage token represents both that token and how the token relates to the question. A token that answers the question will have a hidden state shaped by the question's content, which the linear start/end classifiers can then detect.

SQuAD 1.1 guaranteed that every question had an answer in the passage. SQuAD 2.0 added roughly 50,000 adversarially crafted unanswerable questions, plausible-looking enough that a shallow model might confabulate an answer. Questions like "What year was Einstein born in Paris?" are unanswerable from a passage that mentions Einstein's birth in Ulm, Germany. A model that always extracts some span will get these wrong by design.

This change was motivated by a practical concern: deployed QA systems must know when to abstain. A system that always produces an answer is dangerous if the answer is fabricated. SQuAD 2.0 forced models to reason about whether the question is even answerable from the given passage, not just which span is most likely.

Models trained on SQuAD 2.0 learn to produce a "no answer" prediction when confident. The standard approach adds an "answerability score" (the probability that the question is answerable at all) computed from the [CLS] token or as a separate classification head. If the answerability score falls below a threshold, the model outputs "No answer".

This threshold controls a tradeoff: a low threshold produces fewer false "no answer" predictions but more hallucinated spans; a high threshold produces more "no answer" predictions, even for answerable questions. Calibrating this threshold for a specific application requires evaluation on a held-out set representative of the deployment distribution. In a medical QA system where a wrong answer is more dangerous than no answer, you set the threshold high. In a general search assistant where users prefer a possibly wrong answer over no answer, you set it lower.

The answerability prediction also raises an interesting question about what the model has learned. To correctly identify an unanswerable question, the model must understand what information is in the passage and what information is missing. This is harder than simply finding a matching span, and models that score well on SQuAD 2.0 have developed more reliable passage comprehension than their SQuAD 1.1-only counterparts.

In practice, you rarely have the correct passage pre-selected. In open-domain extractive QA, the system must first retrieve relevant passages from a large corpus (Wikipedia, a document collection, the web) before running the reader.

This retrieval step uses one of several strategies:

• Sparse retrieval: TF-IDF or BM25 matches query terms to document terms. Fast and interpretable, but fails on vocabulary mismatch (the question uses "death" where the document uses "passed away"). BM25 is the standard sparse baseline, and it remains competitive for many retrieval tasks despite its simplicity.
• Dense retrieval: Encode question and passages using a bi-encoder neural network; retrieve by nearest-neighbor search in embedding space. Handles synonymy and paraphrase, but requires building a vector index. Dense retrieval became dominant after Dense Passage Retrieval (DPR) was introduced in 2020.
• Hybrid retrieval: Combine sparse and dense scores. Often outperforms either alone, since sparse retrieval captures exact keyword matches while dense retrieval captures semantic similarity; combining both recovers the strengths of each approach.

The full pipeline is: retrieve top-$k$ passages, run the reader on each passage independently, then select the best answer span across all passages using the span confidence scores. The reader does not know which passage came from which document; it simply processes each passage as an independent context and reports which span it finds most likely to be the answer.

One practical concern is passage length. BERT-based readers typically cap at 512 tokens. Wikipedia passages are usually chunked into 100-word blocks for retrieval to ensure they fit within this window. Longer documents require sliding-window approaches, where the same document is processed in overlapping windows and the best span across all windows is selected.