Question Answering › Summary

Question answering spans a wide design space, from simple span extraction to complex multi-hop reasoning with retrieval and generation. The key ideas to carry forward:

• Extractive QA frames answer finding as span selection, predicting start and end token indices in a passage. BERT-based models trained on SQuAD achieve near-human performance on the benchmark and remain widely used for grounded, low-latency QA.
• Generative QA generates free-form answers, allowing synthesis across documents and natural phrasing. Instruction-tuned LLMs handle this with prompting, but hallucination requires careful management.
• Open-domain QA adds a retrieval step before reading, using sparse (BM25) or dense (DPR) retrieval to fetch relevant passages from large corpora. Dense retrieval handles vocabulary mismatch; Fusion-in-Decoder enables multi-passage synthesis.
• Evaluation uses Exact Match and F1 for short answers, ROUGE or BERTScore for longer outputs, and LLM-as-judge for complex generative responses. Retrieval quality should be evaluated separately from reading quality.
• Multi-hop QA requires combining information across documents; chain-of-thought prompting and iterative retrieval are the leading approaches.
• Limitations include retrieval bottlenecks, hallucination in generative systems, evaluation gaps between benchmarks and deployment, and domain adaptation challenges. Reliable QA systems require ongoing evaluation and maintenance, rather than one-time deployment.

The next chapter extends these ideas to information extraction: rather than answering natural language questions, we structure the information in text into typed entities, relations, and events, a more explicit form of knowledge extraction that powers knowledge graphs and structured databases.