Contrastive Learning for Retrieval: InfoNCE & DPR Guide › Hard Negative Mining

Hard negatives are passages that are superficially similar to the query or the positive passage but are irrelevant. They force the model to learn beyond surface-level matching and develop a deeper understanding of what makes a passage relevant to a query. If in-batch negatives teach the model to distinguish apples from oranges, hard negatives teach it to distinguish a Granny Smith from a Honeycrisp, both of which are apples but differ in ways that matter.

Negatives fall on a spectrum from trivial to hard, and the position of a negative on this spectrum determines how much the model can learn from it:

• Trivial negatives: Completely unrelated passages. "What is photosynthesis?" paired with a passage about medieval architecture. The model separates these easily after minimal training because they discuss different topics with unrelated vocabulary and meaning. These examples contribute negligible gradients.
• Medium negatives: Topically related but non-answering passages. "What is photosynthesis?" paired with a passage about chloroplasts that doesn't explain the process. These share vocabulary and topic but differ in specificity. The model must learn that topical overlap is necessary but not sufficient for relevance.
• Hard negatives: Passages that seem highly relevant but don't answer the query. "What is photosynthesis?" paired with a passage that defines cellular respiration using similar terminology (e.g., mentioning "energy conversion in plant cells"). These require semantic understanding to distinguish, because the surface-level features of vocabulary, sentence structure, and topic all point toward relevance, yet the passage fails to provide the information the query seeks.

The key insight is that training on mostly trivial negatives wastes gradient updates. The loss is already near zero for these examples, so the model learns nothing from them. Hard negatives, by contrast, produce meaningful gradients that push the model to learn the fine-grained distinctions that matter for retrieval quality. They are the examples that sit near the decision boundary between "relevant" and "irrelevant" in the embedding space, and it is precisely at this boundary that the model's representations need the most refinement.

There are several practical approaches to finding hard negatives, each exploiting a different notion of "surface similarity" to identify passages that will challenge the model:

BM25 negatives. Use BM25 (which we covered in Part II) to retrieve the top-$k$ passages for each query, then exclude the known positives. BM25 finds passages with high lexical overlap, which are topically related but often not the right answer. These form excellent hard negatives because they have exactly the surface-level features a naive model might rely on: shared keywords, matching entities, and overlapping phrases. By training the model to distinguish these BM25-retrieved passages from the true positive, you force it to learn that lexical overlap alone does not constitute semantic relevance, pushing it toward a deeper understanding of meaning.

Model-based negatives. Use the retrieval model itself (or a previous checkpoint) to find its highest-scoring non-relevant passages. These are the passages the model currently "thinks" are relevant but aren't, precisely the errors the model most needs to learn from. This creates an iterative training loop: train the model, use it to mine hard negatives, retrain on those negatives, and repeat. Each iteration produces harder negatives than the last, because the model has corrected its previous mistakes and the remaining errors are increasingly subtle. This approach is sometimes called "self-mining" or "iterative hard negative mining," and it can yield significant improvements over static negative sets, though it comes with the computational cost of periodically re-encoding the corpus.

Cross-query negatives. The positive passage for a closely related query can serve as a hard negative. If query $q_1$ is "When was the Eiffel Tower built?" and $q_2$ is "Who designed the Eiffel Tower?", the positive passage for $q_2$ might discuss Gustave Eiffel's design process without mentioning the construction year, making it a hard negative for $q_1$. These negatives are particularly valuable because they are relevant to the topic but address a different information need, forcing the model to distinguish between queries that share a subject but differ in intent.

Gold negatives from annotations. Some datasets include explicit negative annotations where annotators mark passages as "related but not relevant." These are the highest-quality hard negatives but the most expensive to obtain, because they require human judgment to identify passages that are deceptively similar to the true positive. When available, they provide the clearest training signal, as there is no risk of mislabeling.

Using only hard negatives can hurt training, a counterintuitive result that reveals an important subtlety of the contrastive learning framework. If every negative is nearly indistinguishable from the positive, the loss surface becomes very noisy and the model may struggle to converge. The gradients from each training step will be large and highly variable, pushing the model in inconsistent directions. Worse, some mined hard negatives may be false negatives, passages that are relevant but aren't labeled as such. Training the model to push these away degrades performance, because the model is being rewarded for making incorrect predictions.

The false negative problem is especially acute with model-based mining. If the current model assigns a high score to a passage, and that passage is excluded from the positive set simply because it wasn't in the original annotations, there is a real chance the passage is relevant. Pushing it away wastes a gradient step and actively corrupts the learned representation. This is why some recent work explores techniques for detecting and mitigating false negatives, such as denoising heuristics or soft labels that give partial credit to borderline passages.

The best practice is to mix hard negatives with random (in-batch) negatives. DPR, for example, uses one BM25 hard negative per query combined with in-batch negatives from the rest of the batch. This gives the model a mix of easy examples (for stable gradients and smooth convergence) and challenging examples (for learning fine distinctions and pushing the decision boundary). The easy negatives provide a reliable "floor" of gradient signal that keeps the model moving in the right direction, while the hard negatives provide the "ceiling" of information that drives the model toward high-quality representations.

Let's visualize how different negative strategies affect the difficulty of the contrastive task: