Hybrid Search: BM25 and Dense Retrieval Combined › Limitations and Impact

Hybrid search is not a free lunch. Combining two retrieval systems doubles the infrastructure: you must maintain two indices, execute two retrieval operations per query, and implement fusion logic that is correct under all edge cases (empty ranked lists, ties, score normalization edge cases). For small-scale applications or corpora with clear lexical structure, a well-tuned BM25 system alone may outperform a poorly-configured hybrid system. The additional complexity pays off only when the two systems have complementary coverage on your specific query distribution.

The latency cost, while often manageable through parallelization, is real. If your retrieval must complete in under 50 milliseconds and your dense retrieval alone takes 40 milliseconds, adding BM25 without careful parallel execution will breach latency budgets. Hybrid search also shifts the debugging challenge: when results are wrong, it is now unclear whether the failure originates in the BM25 component, the dense component, or the fusion logic. A document that deserved to rank first might have ranked 3rd in BM25 (low BM25 score) and 3rd in dense (moderate cosine similarity), giving it a low combined score even though it was the best match. Diagnosing this requires inspecting both individual system outputs, which adds operational overhead.

Weighted score combination introduces an additional fragility: the normalization is sensitive to the retrieval pool size. If you retrieve top-100 from each system before fusing, the min and max scores used for normalization are computed over 100 results. If you change to top-50, the normalization range changes, and the same document receives a different normalized score even though nothing else changed. This coupling between retrieval depth and score normalization is a subtle source of inconsistency in production systems that switch retrieval depths dynamically. RRF does not have this problem because it uses only rank positions.

Despite these challenges, hybrid search has become a standard architectural pattern in production RAG systems because the recall gains are consistent and the tradeoffs are well understood. The BEIR benchmark, a heterogeneous collection of information retrieval datasets, has repeatedly shown that no single retrieval method dominates across all domains. Hybrid search is the practical response to this fragmentation: rather than betting on one paradigm, you hedge across both. The incremental infrastructure cost is justified by more consistent results across diverse query types.

The impact on downstream RAG performance is particularly significant. As we explore in the upcoming Reranking chapter, the generator's answer quality depends critically on whether the correct document appears in the retrieved context at all. A failure to retrieve the relevant document cannot be corrected by a better reranker or a more capable generator. By increasing recall, hybrid search raises the quality ceiling for the entire RAG pipeline.