Multi-Vector Retrievers (2021) › The Problem

Single-vector dense retrieval methods like DPR had achieved impressive results on semantic similarity tasks, but they faced fundamental limitations when deployed in real-world information retrieval scenarios. The core issue was information loss: compressing an entire query or document into a single vector required discarding fine-grained information about token sequences and their relationships. While this compression enabled efficient retrieval through simple vector similarity computations, it reduced retrieval precision, particularly for queries requiring exact term matching or several distinct relevance criteria.

Consider a query like "What year did the Berlin Wall fall?" In a single-vector system, this entire question would be encoded as one dense vector, and documents would be encoded as single vectors as well. The system would retrieve documents based on overall semantic similarity between the query vector and document vectors. However, this approach might retrieve documents about Cold War history in general, or documents about Berlin's architecture, or documents discussing political walls metaphorically, even if they don't contain the specific term "Berlin Wall" or the year 1989. The semantic similarity signal is valuable, but it's too coarse-grained for many retrieval tasks. The system cannot easily enforce that retrieved documents contain specific terms, phrases, or entities mentioned in the query, because all matching happens at the aggregated vector level.

Traditional sparse retrieval methods like BM25 operated at the opposite extreme. They matched individual query terms to document terms, enabling precise control over which terms must appear in retrieved documents. BM25 could easily handle queries requiring exact matches for specific entities, technical terms, or phrases. However, BM25 struggled when queries and documents used different wording. A query about "automobile accidents" wouldn't match documents about "car crashes" unless both terms appeared explicitly. The semantic understanding that made dense methods powerful was entirely absent from sparse approaches.

Beyond the granularity problem, single-vector dense retrieval systems faced computational challenges that limited their scalability. Encoding each document as a single vector required storing one vector per document, which was manageable, but the encoding process itself was computationally expensive, especially for long documents that needed to be truncated or processed in segments. More fundamentally, the single-vector representation forced the system to make an early commitment about what information was most important in a document. For a long document covering multiple topics, the single vector had to capture the most salient information, potentially losing details that might be relevant for specific queries. This compression artifact created a ceiling on retrieval performance, particularly for long-form documents or documents with diverse content.

The retrieval scoring mechanism in single-vector systems also had inherent limitations. Computing relevance as cosine similarity between query and document vectors meant that all query terms contributed equally to the final score, and all document terms contributed equally. There was no way to weight the importance of specific query terms or to handle queries where some terms were more critical than others. For instance, in the query "Who invented the telephone?" the term "telephone" is more important than "the," but single-vector systems couldn't easily reflect this distinction. The uniform aggregation meant that queries with many terms might dilute the signal of the most important terms, and queries with few terms might not have enough signal to reliably match relevant documents.

These limitations became particularly apparent when comparing dense single-vector retrieval to traditional sparse methods on tasks requiring precision. While DPR and similar systems achieved strong performance on question answering datasets where semantic similarity determined relevance, they often underperformed on tasks requiring exact term matching, entity retrieval, or queries with specific technical vocabulary. The field needed retrieval systems that could combine the semantic understanding of dense methods with the precision and flexibility of sparse methods, operating at a level of granularity that preserved both fine-grained matching capabilities and semantic relationships.