Question Answering › Open-Domain QA

Open-domain QA (ODQA) refers to answering questions about virtually any topic, without a pre-selected passage. The system must search a large corpus (typically Wikipedia or the web) to find relevant information before answering.

The challenge has two parts: finding the right information, and extracting or generating the answer from it. Getting either step wrong cascades into a wrong final answer. A perfect reader on the wrong passage returns a wrong answer. A perfect passage in the corpus that was never retrieved might as well not exist. This tight coupling between retrieval and reading quality is what makes ODQA harder than closed-domain QA and why the field has invested so heavily in both retrieval architectures and retrieval-reader joint training.

The classic ODQA architecture, formalized by Danqi Chen et al. (2017) in "Reading Wikipedia to Answer Open-Domain Questions" (DrQA), separates the problem into:

1. Document retrieval: Given question $q$, retrieve top-$k$ relevant passages $\{p_1, \ldots, p_k\}$ from a corpus.
2. Machine reader: Given $q$ and each passage $p_i$, extract or generate the answer $a$.
3. Answer selection: If multiple passages produce candidate answers, select the best one (e.g., by confidence score or answer frequency).

DrQA used TF-IDF retrieval and a BiDAF-based reader. Modern implementations replace both stages with neural alternatives: dense passage retrieval (DPR) for retrieval, and BERT/T5 for reading.

The retrieve-then-read pipeline makes each stage independently optimizable. You can improve retrieval without touching the reader, and vice versa. The modular structure also makes the system transparent: you can inspect which passages were retrieved and why. This transparency is valuable in deployed systems, where understanding failure modes requires knowing whether an incorrect answer came from retrieving the wrong passage (retrieval error) or from misreading the correct passage (comprehension error). The two types of failure require different fixes.

Answer selection from multiple passages introduces an additional aggregation challenge. If three passages each produce a candidate answer and the model returns different spans from each, how do you pick the final answer? The simplest approach uses the reader's confidence score: take the span with the highest score regardless of which passage it came from. A more sophisticated approach uses answer voting: if multiple passages yield the same answer string, that answer is weighted more highly. Answer voting provides a form of cross-passage consistency checking that improves reliability, especially on factoid questions where the same fact appears in many documents.

Dense Passage Retrieval (DPR), introduced by Karpukhin et al. (2020), replaces sparse TF-IDF with dense vector similarity. Two BERT encoders (one for questions, one for passages) are trained so that the question embedding and the corresponding answer passage embedding are close in vector space:

$$\text{sim}(q, p) = E_Q(q)^\top E_P(p)$$

where:

• $E_Q(q) \in \mathbb{R}^d$: the dense embedding of the question, produced by the question encoder (a fine-tuned BERT model)
• $E_P(p) \in \mathbb{R}^d$: the dense embedding of passage $p$, produced by the passage encoder (a separate fine-tuned BERT model)
• $\top$: the transpose, making this a dot product between two vectors of dimension $d$ (typically 768 for BERT-base)

Training uses contrastive learning. For each question, one positive passage (contains the answer) and multiple negative passages (don't contain the answer) are provided. The loss encourages the question to be closer to the positive passage than to all negatives:

$$\mathcal{L}_{\text{DPR}} = -\log \frac{\exp(\text{sim}(q, p^+))}{\exp(\text{sim}(q, p^+)) + \sum_{i=1}^{m}\exp(\text{sim}(q, p_i^-))}$$

where:

• $p^+$: the positive passage, i.e., the passage known to contain the answer to question $q$
• $p_1^-, \ldots, p_m^-$: the $m$ negative passages that do not contain the answer (typically sampled from the corpus or from other questions in the same batch)
• The denominator sums the similarity to the positive and all negatives, normalizing the numerator into a probability

This is the InfoNCE (contrastive) loss: it maximizes the log-probability of the positive passage relative to all negatives. Intuitively, it pushes the question embedding toward the positive passage and away from all negative passages in the shared embedding space.

The choice of negative passages matters enormously for DPR training quality. Random negatives (passages sampled at random from the corpus) are easy: the passage about "apple orchards" is obviously not the answer to "What is the capital of France?" Hard negatives are more useful: passages that are superficially relevant (mention France or mention a capital city) but do not contain the answer. Training with hard negatives forces the model to distinguish between passages that mention the right topic and passages that contain the actual answer, which is a much more useful distinction for retrieval.

At inference time, all passages in the corpus are encoded once and stored in a vector index (FAISS is the standard choice). Retrieval becomes a maximum inner product search: find the passage embeddings with the highest dot product to the query embedding. This is orders of magnitude faster than re-encoding on every query. A corpus of 21 million Wikipedia passages encoded with DPR can be searched in milliseconds using approximate nearest-neighbor search, even on CPU.

DPR significantly outperforms BM25 on standard ODQA benchmarks, particularly for questions where the relevant passage uses different vocabulary than the question. A question about "automobile manufacturing" retrieves passages about "car production" because the dense embeddings capture semantic similarity, not just surface string overlap.

A limitation of the retrieve-then-read pipeline is that the reader processes each passage independently. If the answer requires information from multiple passages (e.g., one passage names the person, another gives their dates), span extraction fails and even generative models may struggle without seeing all relevant passages together.

Fusion-in-Decoder (FiD), introduced by Izacard and Grave (2021), addresses this by processing all retrieved passages jointly. Each passage is independently encoded by a T5 encoder, producing a sequence of encoder hidden states. The decoder then attends over the concatenated hidden states from all passages simultaneously:

$$\mathbf{H}_{\text{fused}} = [\mathbf{H}_{p_1}; \mathbf{H}_{p_2}; \ldots; \mathbf{H}_{p_k}]$$

where:

• $\mathbf{H}_{p_i} \in \mathbb{R}^{L \times d}$: the matrix of encoder hidden states for passage $p_i$, with $L$ being the passage length and $d$ the hidden dimension
• $[\cdot ; \cdot]$: row-wise concatenation, producing a combined matrix of shape $(k \cdot L) \times d$
• $k$: the number of retrieved passages

The decoder cross-attention operates over $\mathbf{H}_{\text{fused}}$, enabling it to synthesize information from multiple passages when generating the answer. This is the key architectural move: the encoder processes passages independently (which is computationally efficient and allows parallel encoding), but the decoder sees all encoder outputs simultaneously (which enables multi-passage synthesis).

FiD achieved state-of-the-art results on Natural Questions and TriviaQA at the time of publication, showing that reading more passages jointly is better than reading them separately and aggregating. The gains are largest on questions where the answer requires combining evidence from multiple sources, confirming the design motivation.

The computational cost scales linearly with the number of passages $k$: encoding $k$ passages requires $k$ forward passes through the encoder. This is manageable because encoder passes can be parallelized, but inference latency grows with $k$. In practice, $k = 100$ retrieved passages is common for FiD, which would be prohibitive for a pipeline that runs each passage through a full reader independently.