Hybrid Search: BM25 and Dense Retrieval Combined › Why Hybrid Search Works

The core empirical finding across dozens of retrieval benchmarks is that BM25 and dense retrieval have complementary recall. Documents missed by BM25 are often found by dense retrieval, and vice versa. When you combine the two, recall at depth K (the fraction of relevant documents appearing in the top K results) increases substantially over either system alone. This is not a theoretical claim: it has been validated repeatedly on the BEIR benchmark, the MS MARCO dataset, and numerous domain-specific retrieval evaluations.

To understand why this complementarity is so consistent, it helps to think about what each system is optimized for. BM25 is optimized for exact lexical evidence: it finds documents that share the surface tokens of the query. Dense retrieval is optimized for semantic similarity: it finds documents whose meaning aligns with the query's meaning. These objectives differ. A document about "cardiac arrest management" contains the semantic content relevant to a query about "heart attack treatment" but almost none of the lexical tokens. A document containing a product serial number "QX-7721" contains the lexical tokens relevant to that query but no semantic relationship to any nearby concept. The systems see different aspects of relevance.

The key insight is that neither system is a noisy version of the other. If BM25's failures were just random errors and dense retrieval's failures were also random errors, combining them might help slightly by averaging out noise. But the failures are systematic and complementary: BM25 consistently fails on semantic queries and dense consistently fails on lexical queries. This systematic pattern means that every document in BM25's failure set is a candidate for dense retrieval's success set, and vice versa. The two failure modes partition the query space, and together the two systems cover the entire partition.

More precisely:

• Dense retrieval excels at semantic paraphrase queries, synonym-heavy queries, and cross-lingual queries
• BM25 excels at queries containing rare terms, entity names, technical codes, and queries where exact string matching matters
• Neither dominates consistently across all query types

Hybrid search is not about one system helping the other. It is about both systems searching independently, then intelligently combining their separate ranked lists or scores into a single, more complete result. Think of it like two independent experts reviewing the same corpus. Expert A (BM25) has a photographic memory for exact words and can instantly identify any document containing a specific phrase. Expert B (dense retrieval) has a deep understanding of meaning and can identify documents that address the same concept even when worded completely differently. Neither expert is redundant. The combined recommendation of both experts, when appropriately weighted, will be more complete than either recommendation alone.

The chart confirms the complementarity argument visually. For lexical queries (exact product codes, rare identifiers), BM25 retrieves 70% of relevant documents on its own while dense retrieval alone captures only 5%. For semantic queries (paraphrases, synonym-based questions), the situation reverses: dense retrieval captures 70% and BM25 alone captures only 5%. The "Both systems" bar is documents that either system would have found. Hybrid search captures the union of the BM25-only and dense-only bars in addition to the overlap, making the combined recall the sum of all three columns.