BM25 › A Worked Example
Language AI HandbookAll chapters BM25 › A Worked ExampleOne section · Sections in this chapter5 / 14 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
With the formula understood, let's trace through a concrete example to see how all the pieces fit together. We'll score three short documents against a simple query, computing each component step by step.
Our corpus consists of several documents:
- D1: "the cat sat on the mat" (6 words)
- D2: "the cat sat on the cat mat" (7 words)
- D3: "the dog ran in the park" (6 words)
And our query is: "cat mat"
Notice that D1 and D2 both contain our query terms, but D2 mentions "cat" twice. D3 contains neither term. Let's see how BM25 handles these differences.
Step 1: Collection Statistics
Before scoring any document, we need collection-level statistics. These establish the baseline against which individual documents are compared:
documents = [
"the cat sat on the mat",
"the cat sat on the cat mat",
"the dog ran in the park",
"the bird flew over the house",
"the fish swam in the pond",
"the rabbit hopped through the field",
"the horse trotted down the road",
]
query = "cat mat"
# Tokenize
doc_tokens = [doc.lower().split() for doc in documents]
query_tokens = query.lower().split()
# Collection statistics
N = len(documents) # Total documents
doc_lengths = [len(tokens) for tokens in doc_tokens]
avgdl = sum(doc_lengths) / N
# Document frequency for each term
from collections import Counter
doc_freq = Counter()
for tokens in doc_tokens:
for term in set(tokens):
doc_freq[term] += 1Number of documents (N): 7 Document lengths: [6, 7, 6, 6, 6, 6, 6] Average document length (avgdl): 6.14 Document frequencies for query terms: 'cat': appears in 2 document(s) 'mat': appears in 2 document(s)
We have 7 documents with an average length of about 6 words. "Cat" appears in 2 of the 7 documents, and "mat" appears in 2 as well. These statistics feed into our IDF and length normalization calculations.
Step 2: Computing IDF
Now let's compute the IDF for each query term. Remember, IDF measures how discriminative a term is. Terms appearing in fewer documents get higher IDF:
import math
def compute_idf(term, N, doc_freq):
n = doc_freq.get(term, 0)
return math.log((N - n + 0.5) / (n + 0.5))
idf_scores = {term: compute_idf(term, N, doc_freq) for term in query_tokens}IDF scores: 'cat': log((7 - 2 + 0.5) / (2 + 0.5)) = 0.7885 'mat': log((7 - 2 + 0.5) / (2 + 0.5)) = 0.7885
Both terms appear in 2 out of 7 documents. Their IDF values reflect how specific they are relative to this collection. With a larger corpus, terms appearing in only 2 out of thousands of documents would receive much higher IDF values. This small corpus example illustrates the mechanics, but the dynamics play out more clearly at scale.
Step 3: Scoring Each Document
Now we combine everything: IDF, saturated term frequency, and length normalization. For each query term in each document, we compute the contribution and sum them up:
def bm25_score(doc_tokens, query_tokens, doc_freq, N, avgdl, k1=1.2, b=0.75):
doc_len = len(doc_tokens)
term_freq = Counter(doc_tokens)
score = 0
score_breakdown = []
for term in query_tokens:
if term not in term_freq:
continue
tf = term_freq[term]
idf = compute_idf(term, N, doc_freq)
# BM25 term score
numerator = tf * (k1 + 1)
denominator = tf + k1 * (1 - b + b * (doc_len / avgdl))
term_score = idf * (numerator / denominator)
score += term_score
score_breakdown.append(
{"term": term, "tf": tf, "idf": idf, "term_score": term_score}
)
return score, score_breakdown
# Score each document
results = []
for i, tokens in enumerate(doc_tokens):
score, breakdown = bm25_score(tokens, query_tokens, doc_freq, N, avgdl)
results.append(
{
"doc_id": i + 1,
"score": score,
"breakdown": breakdown,
"length": len(tokens),
}
)BM25 Scores (k1=1.2, b=0.75):
Document 2: "the cat sat on the cat mat"
Length: 7 (avgdl = 6.14)
Total Score: 1.7891
'cat': TF=2, IDF=0.7885, contribution=1.0432
'mat': TF=1, IDF=0.7885, contribution=0.7459
Document 1: "the cat sat on the mat"
Length: 6 (avgdl = 6.14)
Total Score: 1.5921
'cat': TF=1, IDF=0.7885, contribution=0.7960
'mat': TF=1, IDF=0.7885, contribution=0.7960
Document 3: "the dog ran in the park"
Length: 6 (avgdl = 6.14)
Total Score: 0.0000
Document 4: "the bird flew over the house"
Length: 6 (avgdl = 6.14)
Total Score: 0.0000
Document 5: "the fish swam in the pond"
Length: 6 (avgdl = 6.14)
Total Score: 0.0000
Document 6: "the rabbit hopped through the field"
Length: 6 (avgdl = 6.14)
Total Score: 0.0000
Document 7: "the horse trotted down the road"
Length: 6 (avgdl = 6.14)
Total Score: 0.0000Let's visualize the score breakdown for each document to see how each query term contributes:
The visualization makes the scoring dynamics clear:
- D2 wins despite being the longest document. Why? It contains "cat" twice, and the saturation effect hasn't fully kicked in at TF=2. The extra occurrence still provides meaningful additional score.
- D1 comes second with both query terms appearing once each. It's slightly shorter than average, which gives it a small boost.
- D3 scores zero because it contains neither query term. No amount of length normalization or parameter tuning can help a document that lacks the query terms entirely.
The negative scores might seem strange, but they're a consequence of our small corpus where both query terms are common relative to the collection size. In practice, with larger corpora and more discriminative query terms, you'd see positive scores for matching documents. What matters is the relative ranking, and BM25 correctly identifies D2 as most relevant.