Multi-Vector Retrievers (2021) › Applications and Impact

Multi-vector retrieval systems found immediate application in question answering and information retrieval tasks where both semantic understanding and precise matching were important. The fine-grained matching capability made these systems particularly effective for queries containing specific entities, technical terms, or exact phrases that needed to appear in retrieved documents. At the same time, the semantic matching enabled retrieval of documents that discussed related concepts even when they used different terminology. This dual capability proved valuable for open-domain question answering, where queries might mix semantic concepts with specific factual details requiring exact matches.

In question answering systems, multi-vector retrievers enabled more accurate passage retrieval by matching individual question terms to relevant document passages. A question like "What year did Einstein publish his theory of special relativity?" could match documents containing "Einstein" in one part, "special relativity" in another, and "1905" or "theory" in yet another, aggregating evidence from across the document rather than requiring exact phrase matches. This flexibility was particularly valuable for long documents where relevant information might be scattered across multiple passages. The token-level matching allowed systems to find documents where multiple query aspects were present, even if they weren't co-located.

Beyond question answering, multi-vector retrieval improved performance on tasks requiring entity retrieval, fact verification, and technical document search. In entity retrieval, queries often contained specific entity names that needed to match exactly, while also requiring semantic understanding of what type of information about the entity was being requested. Multi-vector systems could match entity names precisely while semantically matching the information request, handling queries like "When was the first iPhone released?" where "iPhone" needed exact matching but "first iPhone released" required semantic understanding. This combination of lexical and semantic matching was difficult to achieve with either pure sparse or pure dense methods alone.

The retrieval architecture also enabled new capabilities in conversational and interactive retrieval systems. By operating at the token level, multi-vector systems could handle queries of varying length and complexity more gracefully than single-vector systems. Short queries with few tokens could still retrieve relevant documents by matching those tokens precisely, while long queries with many tokens could aggregate evidence from multiple matches. The token-level granularity also made it easier to explain retrieval decisions, as systems could identify which query tokens matched which document tokens. This provided more interpretable retrieval results than single-vector systems where matching happened in an opaque high-dimensional space.

The influence of multi-vector retrieval extended to the design of subsequent retrieval systems, which increasingly incorporated hybrid approaches combining multiple levels of matching granularity. Systems began combining sparse retrieval, single-vector dense retrieval, and multi-vector retrieval, using different methods for different query types or stages of retrieval. The multi-vector paradigm also influenced the design of embedding models specifically trained for retrieval, as researchers recognized the value of token-level representations that could support flexible matching strategies. The architectural principles introduced in multi-vector retrieval, particularly late interaction and token-level contextualized matching, became standard components in state-of-the-art retrieval systems.