BM25: Probabilistic Ranking for Search

By the early 1990s, information retrieval researchers had spent decades studying how to rank documents by relevance. TF-IDF (term frequency-inverse document frequency) was widely used, but its linear treatment of term counts and its sensitivity to document length produced avoidable ranking errors. Researchers wanted a scoring model that gave repeated terms diminishing returns. It also needed to account for term rarity and document length.

BM25 emerged from probabilistic information retrieval work led by Stephen Robertson and Karen Spärck Jones at the University of London. It combined inverse document frequency with term-frequency saturation and document-length normalization. These additions produced a practical ranking function whose components could be interpreted through a probabilistic account of relevance.

BM25 made three ideas easy to use in one scoring function: probabilistic relevance, document-length normalization, and diminishing returns from repeated terms. The function remains a common lexical-retrieval baseline and is often used alongside neural rankers or in retrieval-augmented generation systems.

TF-IDF multiplies a term's frequency in one document by a measure of how rare that term is across the collection. A term that occurs often in a document but rarely elsewhere receives a high weight. The basic formulation, however, does not saturate repeated occurrences or normalize document length in the way BM25 does.

One limitation was the linear treatment of term frequency: a document containing a query term ten times could receive twice the term-frequency contribution of one containing it five times. Relevance rarely grows at that rate. Once a document is clearly about machine learning, another ten mentions of the phrase add little evidence.

Document length created another bias. Longer documents have more opportunities to contain query terms, so a 50-page technical report might outrank a focused two-page summary even when the summary better answers the query.

TF-IDF also uses inverse document frequency as a heuristic rather than deriving the full score from a relevance model. That makes its behavior useful in practice but harder to connect to explicit assumptions about relevant and non-relevant documents.

Term weights also depend on collection statistics rather than an interpretation of the query. In "machine learning algorithms for natural language processing," TF-IDF can weight rare terms more heavily, but it does not recognize "machine learning" or "natural language processing" as phrases with meanings beyond their individual words.

BM25 addressed these limitations within a probabilistic framework. "BM" stands for "Best Matching," while "25" indicates the 25th iteration of the model developed by Robertson and Spärck Jones.

BM25 follows the probabilistic ranking principle: documents should be ordered by their estimated probability of relevance. Its scoring function approximates this ordering with contributions for term rarity and saturated term frequency, adjusted for document length.

For a query $Q$ containing terms $q_1, q_2, \ldots, q_n$, the BM25 score for document $D$ is:

The $\text{IDF}(q_i)$ term assigns more weight to terms that occur in fewer documents. The fraction containing $f(q_i, D)$ saturates as term frequency rises, preventing repeated occurrences from increasing the score without bound.

The normalization factor addresses document-length bias. The parameter $b$ controls its strength: $b = 0$ applies no length normalization, while $b = 1$ applies full normalization. The ratio $\frac{|D|}{\text{avgdl}}$ compares the current document's length with the collection average.

The parameter $k_1$ controls how quickly term frequency saturates. When $k_1 = 0$, term frequency does not affect the contribution beyond presence; larger values delay saturation and give repeated occurrences more influence.

BM25 is derived from probabilistic information retrieval theory and extends the binary independence model. Its assumptions connect the scoring terms to a model of relevant and non-relevant documents rather than only to a similarity heuristic.

The probabilistic derivation begins with the assumption that we can model the probability of a document being relevant given a query. Using Bayes' theorem, this probability can be expressed as:

Computing this probability directly is impractical, so BM25 uses simplifying assumptions to approximate the log-odds of relevance as a sum of term-level contributions. Each contribution reflects how well a term distinguishes relevant from non-relevant documents.

Relevance judgments can be used to tune BM25's parameters for a collection and query distribution. This made the model practical in evaluations and applications where such feedback was available.

BM25 has two main tuning parameters. The value of $k_1$ controls term-frequency saturation, while $b$ controls document-length normalization. Both can be selected for a particular collection and query set.

The $k_1$ parameter controls term-frequency saturation and is often set between 1.2 and 2.0. Lower values saturate quickly, reducing the effect of repeated boilerplate or specifications. Higher values give term counts more influence when repeated occurrences correlate with relevance in the target collection.

The $b$ parameter controls document length normalization, typically ranging from 0.0 to 1.0. When $b = 0$, the algorithm performs no length normalization, which might be appropriate for collections where document length is a meaningful indicator of comprehensiveness. When $b = 1$, the algorithm performs full length normalization, which is generally preferred for most applications where users want the most relevant content regardless of document length.

BM25 often performs reasonably across a range of parameter settings. Collection-specific tuning can improve results, but useful baselines are possible without an exhaustive parameter search.

BM25 was adopted in research benchmarks, search systems, digital libraries, and enterprise retrieval platforms. Its scoring function was inexpensive to compute and generally improved on simpler TF-IDF variants.

BM25 performed well on standard test collections while remaining inexpensive enough for production retrieval. Those properties encouraged its adoption in search engines and other document-ranking systems.

BM25 also weights query terms differently through inverse document frequency. Rare terms can contribute more than common ones, which helps discriminate among documents for longer or more specific queries.

BM25's parameterization also allowed search engines to optimize their ranking for specific types of content. News search engines could tune parameters for the characteristics of news articles, while academic search engines could optimize for scholarly papers. This flexibility made BM25 adaptable to diverse application domains. This contributed to its widespread adoption.

Digital libraries and academic search systems also adopted BM25. Its term weighting works well with technical vocabulary, and its parameters can be tuned against precision and recall on scholarly collections.

Academic search systems using BM25 could now provide more relevant results for complex research queries, helping researchers discover relevant papers and resources more effectively. The algorithm's probabilistic foundation also enabled more sophisticated relevance feedback mechanisms, allowing systems to learn from users' interactions and improve over time.

The impact on academic information retrieval extended beyond search quality. BM25's theoretical foundation provided a solid basis for further research in probabilistic information retrieval, inspiring numerous extensions and improvements. The algorithm became a benchmark against which new ranking methods were evaluated, establishing a standard for experimental evaluation in the field.

Beyond public search engines and academic systems, BM25 found extensive applications in enterprise information management. Organizations with large internal document collections could now implement more effective search capabilities, helping employees find relevant information more efficiently.

The algorithm's parameterization proved particularly valuable in enterprise settings, where different departments might have different types of content and information needs. IT departments could optimize BM25 for technical documentation, while legal departments could tune it for case law and regulatory documents. This flexibility made BM25 adaptable to diverse organizational contexts.

BM25's computational cost made it practical for large internal collections with limited resources. Reasonable default parameters also allowed organizations to deploy a useful baseline before investing in collection-specific tuning.

BM25 relies on simplifying assumptions that limit its effectiveness in some settings. Its treatment of terms as independent lexical matches is especially important when comparing it with later semantic retrieval methods.

The most fundamental limitation of BM25 lies in its assumption of term independence, a simplification that makes the probabilistic calculations tractable but doesn't reflect the reality of how terms interact in natural language. The algorithm treats each query term as contributing independently to the relevance score, ignoring the complex relationships between terms that are central to human language understanding.

This independence assumption becomes particularly problematic for queries involving multiple related terms. Consider a query like "machine learning algorithms for natural language processing." BM25 would score a document based on the individual contributions of "machine," "learning," "algorithms," "natural," "language," and "processing," without considering that the phrase "machine learning" or "natural language processing" might be more meaningful than the sum of its parts. A document containing "machine learning" as a coherent concept might be more relevant than one containing "machine" and "learning" in completely different contexts.

The independence assumption also limits BM25's ability to handle semantic relationships between terms. Synonyms, hyponyms, and other linguistic relationships are invisible to the algorithm, which can only work with exact term matches. This limitation becomes increasingly problematic as document collections grow and users expect more sophisticated understanding of their information needs.

BM25's reliance on exact term matching creates significant limitations in its ability to understand semantic content. The algorithm cannot recognize that "automobile" and "car" refer to the same concept, or that "doctor" and "physician" are synonyms. This limitation becomes particularly problematic in specialized domains where multiple terms might refer to the same concept, or where technical terminology varies across different sources.

The lack of semantic understanding also limits BM25's ability to handle morphological variations. The algorithm treats "run," "runs," "running," and "ran" as completely different terms, even though they represent different forms of the same concept. While stemming and lemmatization can partially address this issue, they introduce their own complexities and don't fully solve the semantic understanding problem.

These limitations become increasingly apparent as users' expectations for search systems grow. Modern users expect search engines to understand their intent, not just match their exact words. While BM25 represented a significant advance over previous approaches, it still operated at the lexical level rather than the semantic level that characterizes human language understanding.

BM25 produces static rankings that don't adapt to individual users or their specific contexts. The algorithm treats all users equally, ranking documents based on the same probabilistic model regardless of the user's background, preferences, or previous interactions with the system. This one-size-fits-all approach limits the algorithm's ability to provide truly personalized search experiences.

The static nature of BM25 rankings also means that the algorithm cannot learn from user behavior or adapt to changing information needs. While the algorithm's parameters can be tuned based on general performance metrics, it cannot incorporate individual user feedback or adapt its ranking strategy based on successful or unsuccessful search sessions.

This limitation becomes particularly problematic in contexts where user preferences and contexts vary significantly. A medical researcher searching for information about a specific condition might have very different relevance criteria than a patient seeking the same information. BM25's inability to adapt to these different contexts limits its effectiveness in diverse user populations.

BM25 remains a baseline for neural ranking and a lexical retriever in some retrieval-augmented generation systems. Later methods often compare against it or combine its exact-term signal with learned semantic representations.

Neural ranking models that emerged in the 2000s and 2010s learned non-linear relationships between queries and documents. Many still used BM25 as a first-stage retriever, a feature, or a baseline because its lexical signal complemented learned representations.

Some neural rankers include BM25 scores as features or re-rank candidates retrieved by BM25. Such systems combine exact lexical matches with semantic signals learned from data.

BM25 exposes term frequency, document length, and term rarity as separate factors. Neural rankers can instead learn how to combine lexical and semantic features from training data.

Retrieval-augmented generation (RAG) systems retrieve documents to supply context for generation. BM25 can serve as the retriever or as a first stage before neural re-ranking.

In a RAG pipeline, BM25 can retrieve candidate documents quickly from a large knowledge base. Neural retrievers offer semantic matching, while BM25 contributes an inexpensive exact-term signal. Production systems may use either method or combine them in hybrid retrieval.

BM25 scores rank candidate documents, but they are not calibrated probabilities. A RAG pipeline may use those rankings directly or combine them with neural retriever and re-ranker scores.

BM25 is a common baseline for evaluating new ranking and retrieval methods. Its implementation and behavior are well understood, so a comparison can show whether a more complex method improves on a strong lexical ranker.

As a benchmark, BM25 provides a stable reference point for comparing retrieval methods. It can also produce a reasonable baseline without extensive tuning, provided that preprocessing and evaluation conditions are held constant.

Because BM25 is widely implemented, researchers can reproduce a familiar baseline across collections. Meaningful comparisons still require reporting preprocessing, parameter choices, and evaluation metrics.

The following questions review BM25's saturation term, length normalization, independence assumption, and modern retrieval uses.

undefined