BM25 › Limitations and Modern Context

BM25 works well in practice, but it has fundamental limitations that become apparent as soon as you push beyond exact keyword matching.

The most significant limitation is vocabulary mismatch. BM25 only matches exact terms. If your query says "car" but the document says "automobile," BM25 won't find the connection. This is not a minor edge case: it's a fundamental restriction of lexical matching that affects every retrieval task. Users phrase queries in their own words, which may not match the words authors chose when writing documents. Synonyms and paraphrases create vocabulary gaps, as do abbreviations and spelling variants that BM25 cannot bridge.

Closely related is the lack of semantic understanding. "Bank of the river" and "bank account" would be treated identically by BM25: it sees the tokens, not the meaning. A query about financial institutions would retrieve documents about riverbanks if they happen to use the word "bank" heavily. This kind of lexical ambiguity is pervasive in natural language, and BM25 has no mechanism to resolve it.

The bag-of-words assumption also discards word order entirely. "Dog bites man" scores the same as "Man bites dog" for a query about dogs and men, despite having very different meanings. Phrase queries are a partial workaround (Elasticsearch supports them with match_phrase), but they require exact contiguous matches and don't generalize to paraphrases.

Finally, BM25 does not capture relationships between query terms. A query for "machine learning" is treated as two independent queries for "machine" and "learning." A document about industrial machinery that also happens to discuss learning processes might score well, even though it's irrelevant to the query's intent.

Despite these limitations, BM25 remains widely used because of several practical advantages:

• Speed: BM25 requires only sparse vector operations over the inverted index, making it extremely fast even on billion-document collections.
• Interpretability: You can explain exactly why a document ranked where it did. The score is a sum of per-term contributions, each traceable back to the formula.
• No training required: BM25 works out of the box on any text collection without labeled examples or GPU compute.
• Handling rare terms: Neural retrievers struggle with rare or unseen terms (out-of-vocabulary words). BM25 handles them naturally since it only requires the term to appear in the index.

Modern systems often combine BM25 with neural retrievers through hybrid search, getting the best of both worlds.

The typical hybrid pipeline works as follows: BM25 retrieves a first-stage set of candidates (say, the top 100 or 1000 documents), then a neural re-ranker like a cross-encoder re-scores those candidates and selects the final top-10. This is much cheaper than running a neural model over the entire collection, while still allowing semantic understanding in the final ranking stage. Elasticsearch and Vespa have native support for hybrid retrieval with BM25 plus dense vectors; Weaviate does as well.

Another emerging pattern is to use BM25 alongside dense retrievers in parallel, then fuse their rankings with a technique called Reciprocal Rank Fusion (RRF). Each system contributes its rankings independently, and the fusion step combines them without needing to calibrate the score scales. This sidesteps the difficult problem of making BM25 scores and cosine similarities comparable.