BM25 (1994) › 1994: BM25 and Probabilistic Ranking
History of Language AIAll chapters BM25 (1994) › 1994: BM25 and Probabilistic RankingOne section · Sections in this chapter1 / 7 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
By the early 1990s, information retrieval researchers had spent decades studying how to rank documents by relevance. TF-IDF (term frequency-inverse document frequency) was widely used, but its linear treatment of term counts and its sensitivity to document length produced avoidable ranking errors. Researchers wanted a scoring model that gave repeated terms diminishing returns. It also needed to account for term rarity and document length.
BM25 emerged from probabilistic information retrieval work led by Stephen Robertson and Karen Spärck Jones at the University of London. It combined inverse document frequency with term-frequency saturation and document-length normalization. These additions produced a practical ranking function whose components could be interpreted through a probabilistic account of relevance.
BM25 made three ideas easy to use in one scoring function: probabilistic relevance, document-length normalization, and diminishing returns from repeated terms. The function remains a common lexical-retrieval baseline and is often used alongside neural rankers or in retrieval-augmented generation systems.