Hybrid Search: BM25 and Dense Retrieval Combined › Fusion Strategies

Given two ranked lists from BM25 and a dense retriever, there are two main ways to combine them: reciprocal rank fusion, which treats both lists as ordinal rankings without using raw scores, and weighted score combination, which directly blends the numeric scores. Each has real tradeoffs, and the choice between them has practical consequences for both performance and operational simplicity.

The challenge of fusion is more subtle than it first appears. The two retrieval systems return results on completely different scales: BM25 scores are unbounded sums of IDF-weighted term frequencies, while cosine similarities are bounded in $[-1, 1]$. A BM25 score of 12.0 and a cosine similarity of 0.85 are not comparable quantities. Any fusion method must either normalize these scores onto a common scale or abandon the raw scores entirely and work only with the rank orderings.

The choice between these strategies is not purely technical. It shows a deeper question about what information you trust. Do you trust the relative ordering of documents more than the absolute magnitude of their scores? Then RRF is your approach. Do you trust that the score magnitudes, after appropriate normalization, carry real information about how much better one document is than another? Then weighted combination may serve you better. Understanding this distinction will help you make better decisions when deploying hybrid search in practice.

Think of the fusion problem as analogous to combining votes from two juries that use different scoring systems. One jury rates cases on a 0-100 scale, the other on a letter grade scale. If you want to combine their verdicts, you have two choices: convert everything to a common scale and average the numbers, or simply note each jury's ranking and combine those ordinal positions. Both approaches work, but they make different assumptions about how real the raw scores are compared to the rank orderings. RRF takes the latter approach; weighted combination takes the former.

Reciprocal rank fusion was introduced by Cormack, Clarke, and Buettcher in 2009. It avoids a basic practical problem: BM25 scores and cosine similarity scores live on completely different numerical scales and cannot be directly added or averaged. BM25 scores might range from 0 to 15 for a given query; cosine similarities might range from 0.7 to 0.95. Adding them raw would let the BM25 system dominate.

To see why this is a problem concretely, imagine a query where BM25 scores are [12.3, 8.7, 6.1, 4.2, 2.0] for its top five documents, and cosine similarities are [0.91, 0.89, 0.87, 0.85, 0.83] for the dense retriever's top five. If you add these raw, the BM25 values (which are an order of magnitude larger) would completely swamp the dense values. The combined ranking would be in effect identical to the BM25 ranking alone. You would have built an expensive two-system pipeline that behaves like a one-system pipeline.

RRF sidesteps this by converting both ranked lists to rank-based scores. The key idea is to replace each document's raw score with a value derived solely from its rank position. For a document $d$ appearing at rank $r$ in retrieval system $s$, its RRF score contribution from that system is:

$$\text{RRF}(d, s) = \frac{1}{k + r_s(d)}$$

where:

• $r_s(d)$: the rank position of document $d$ in system $s$'s ranked list (rank 1 is highest)
• $k$: a smoothing constant, typically set to 60, that prevents the very highest-ranked documents from receiving disproportionately large scores relative to lower-ranked documents

This formula is elegant in its simplicity. Every document gets a score that depends only on its position in the list, not on the magnitude of the score that produced that position. A document ranked first by BM25 gets $1/(60+1) \approx 0.016$ regardless of whether its BM25 score was 15.0 or 0.3. The score scale information is deliberately discarded, and only ordinal position is retained.

The final RRF score for document $d$ across all $n$ retrieval systems accumulates contributions from every system:

$$\text{RRF\_score}(d) = \sum_{s=1}^{n} \frac{1}{k + r_s(d)}$$

where:

• The sum runs over all retrieval systems $s = 1, \ldots, n$ (in hybrid search, typically $n = 2$: BM25 and dense)
• Documents not appearing in a system's ranked list contribute 0 from that system (they are simply omitted)
• Documents appearing high in multiple lists accumulate the largest RRF scores, rewarding consensus across systems

The accumulation structure embodies a clear principle: a document that appears in the top 5 of both systems is more likely to be relevant than a document that appears first in one system and not at all in the other. RRF rewards documents for which both experts agree, and penalizes documents that only one expert endorses. This consensus-seeking property makes RRF naturally reliable: it is hard for a single misbehaving retrieval system to dominate the combined ranking.

Why k = 60?Link Copied

The constant $k = 60$ was found empirically to work well across many retrieval benchmarks. It controls the shape of the rank-score curve. Substituting specific rank values with $k = 60$:

• Rank 1 gives score $1/61 \approx 0.0164$
• Rank 10 gives score $1/70 \approx 0.0143$
• Rank 60 gives score $1/120 \approx 0.0083$
• Rank 1000 gives score $1/1060 \approx 0.00094$

The gap between rank 1 and rank 10 is small: 0.0164 vs 0.0143, a ratio of roughly 1.15. This is intentional. RRF gives credit to many documents, including those below the very top ranks, which helps with diverse query types. Smaller values of $k$ would amplify the difference between rank 1 and rank 10, making the fusion more aggressive in favoring top-ranked documents.

To build intuition for the role of $k$, consider what happens at the extreme values. If $k = 0$, the score for rank 1 is $1/1 = 1.0$ and for rank 2 it is $1/2 = 0.5$, a 2x gap. For rank 10 it is $1/10 = 0.1$, a 10x gap from first place. The first-ranked document would dominate heavily. At $k = 1000$, the scores at ranks 1 through 10 would be nearly identical, hovering around $1/1001$ through $1/1010$. The ranking would be insensitive to rank differences in the top 10. The default $k = 60$ sits in a practical middle ground: sensitive enough to reward top-ranked documents appropriately, but not so aggressive that a single lucky top placement completely overrides everything else.

A Worked RRF ExampleLink Copied

Suppose BM25 returns $[d_3, d_1, d_5, d_2, d_7]$ and the dense retriever returns $[d_1, d_5, d_2, d_4, d_3]$.

With $k = 60$:

Final order: $d_1, d_5, d_3, d_2, d_4, d_7$

Notice that $d_3$ was BM25's top result but placed 5th in dense retrieval. It falls to third overall because $d_5$ ranked well in both systems. $d_4$ appeared only in the dense list and scores lower than all documents with dual coverage. RRF rewards consensus across systems.

This example also illustrates what RRF does to documents that appear in only one list. Document $d_4$ ranked 4th in the dense list, which is a respectable position, but it was absent from BM25 entirely. Document $d_7$ ranked 5th in BM25 but was absent from the dense list. Both receive only single-system credit, which is less than the credit received by any document appearing in both lists. Even $d_2$, which ranked 4th in BM25 and 3rd in dense (not particularly impressive positions in either), receives more total credit than $d_4$ or $d_7$ because it appears in both lists. This consensus preference is the defining property of RRF.

The key insight from this worked example is that RRF does not care about the magnitude of the underlying BM25 or dense scores. Whether $d_3$ had a BM25 score of 15.0 or 1.5, it gets exactly the same RRF contribution: $1/61$. This scale-invariance is what makes RRF the safe default for hybrid search: you never need to worry about one retrieval system's scores dominating because of scale differences, and you never need to estimate normalization parameters from data.

The alternative to RRF is to blend the actual scores directly. This requires that both systems return scores on comparable scales, which usually means normalizing them first. Weighted combination is more expressive than RRF: it lets you say that document A ranked higher than document B and by how much. When the score differences carry real information (a cosine similarity of 0.95 really is substantially better than 0.70), weighted combination can use that information in ways that RRF cannot.

A common approach is min-max normalization applied per query. For each retrieval system, we rescale its raw scores to lie in the range $[0, 1]$ by subtracting the minimum observed score and dividing by the observed score range:

$$\hat{s}_\text{BM25}(d) = \frac{s_\text{BM25}(d) - \min_j s_\text{BM25}(d_j)}{\max_j s_\text{BM25}(d_j) - \min_j s_\text{BM25}(d_j)}$$

where:

• $s_\text{BM25}(d)$: the raw BM25 score for document $d$
• $\min_j s_\text{BM25}(d_j)$: the minimum BM25 score across all retrieved documents $d_j$
• $\max_j s_\text{BM25}(d_j)$: the maximum BM25 score across all retrieved documents $d_j$
• The resulting $\hat{s}_\text{BM25}(d) \in [0, 1]$, with 0 for the lowest-scoring document and 1 for the highest

This normalization is linear, which means it preserves the relative gaps between scores. If one document scored twice as high as another in raw BM25, it will still score twice as high after normalization (assuming the minimum score is 0). It also brings scores from different systems onto the same scale, making direct combination possible.

The same normalization is applied to the dense scores: $\hat{s}_\text{dense}(d) \in [0, 1]$. After normalizing both BM25 and dense scores to $[0, 1]$, the combined score is a weighted sum:

$$\text{hybrid}(d) = \alpha \cdot \hat{s}_\text{dense}(d) + (1 - \alpha) \cdot \hat{s}_\text{BM25}(d)$$

where:

• $\alpha \in [0, 1]$: a mixing hyperparameter controlling how much weight the dense retrieval score receives
• $1 - \alpha$: the complementary weight assigned to the BM25 score
• Setting $\alpha = 1$ gives pure dense retrieval; setting $\alpha = 0$ gives pure BM25; values around $0.5$ to $0.7$ often work well on mixed query benchmarks

The mixing parameter $\alpha$ is the key lever for domain adaptation in weighted fusion. It lets you express a prior belief about how much to trust each retrieval modality. On a corpus of scientific literature where users frequently search by exact paper titles, author names, or technical acronyms, you might set $\alpha = 0.3$, giving 70% weight to BM25. On a customer-facing search system for a knowledge base where users ask open-ended questions in natural language, you might set $\alpha = 0.7$, giving 70% weight to dense retrieval. The ability to express this prior numerically, and to tune it empirically on labeled data, is the main advantage of weighted combination over RRF.

Advantages and DisadvantagesLink Copied

Weighted combination allows more precise tuning when you have query-level signals. If you know a query is lexical (an exact product name), you might increase $1 - \alpha$. If you know a query is semantic (a conceptual question), you might increase $\alpha$. Systems that classify query type before retrieval can dynamically set $\alpha$.

The downside is sensitivity to normalization. Min-max normalization is affected by outliers: if one document has an extremely high BM25 score, it compresses all other scores toward zero. A single highly-matching document (for instance, one that contains every query term many times in a short document) can push all other documents' normalized scores close to zero, effectively neutralizing the contribution of BM25 for that query. Less outlier-sensitive alternatives include score clipping before normalization or using rank-based sigmoid normalization. In contrast, RRF is entirely immune to outliers because it only uses ordinal position.

Beyond these two basic strategies, more advanced fusion methods exist. Learn-to-rank approaches train a lightweight model on top of retrieval features (BM25 score, dense score, document length, query length) to predict relevance. These require labeled training data but can outperform fixed fusion weights materially on specialized domains.

The intuition behind learn-to-rank fusion is that different features matter differently for different query types. A logistic regression or gradient boosted tree trained on relevance labels can learn that BM25 score is highly predictive for short, specific queries but dense score is more predictive for long, natural language questions. No fixed $\alpha$ can capture this query-type-conditional behavior, but a learned model can.

Convex combination with cross-validation, where $\alpha$ is tuned on held-out queries and their relevance judgments, is a practical middle ground that does not require full learn-to-rank infrastructure. If you have even a few hundred labeled query-document pairs from your domain, a cross-validated $\alpha$ sweep over a grid of values (say, 0.0, 0.1, 0.2, ..., 1.0) typically takes seconds and often yields a real improvement over the default $\alpha = 0.5$.