Reranking: Cross-Encoders for Precise Information Retrieval › Worked Example

Let us walk through a concrete reranking scenario in full numerical detail. Suppose you have built a technical support system for a software company. You ask:

"How do I reset my password if I forgot the email associated with my account?"

This query has a specific constraint that distinguishes it from a generic password reset question: the user has forgotten both the email address and the password. The forgotten email address is the crux of the relevance judgment, and it is exactly the kind of subtle distinction that bi-encoders tend to miss.

Step 1: Initial bi-encoder retrieval. Your bi-encoder encodes the query into a 384-dimensional vector and computes cosine similarity against your pre-indexed document embeddings. The system returns these three candidates:

Doc A: "To reset your password, click 'Forgot Password' on the login page. Enter your registered email address, and we will send you a reset link."

Doc B: "Account recovery options include password reset via email, security questions, or contacting support. If you no longer have access to your registered email, use the alternative verification methods available in your account settings."

Doc C: "Email notifications can be configured in your account settings. You can choose to receive updates about password changes, login attempts, and security alerts to any verified email address."

The bi-encoder ranking: Doc A (score 0.89), Doc C (score 0.85), Doc B (score 0.82). Doc A mentions "reset your password" and "email" prominently and matches the query's surface vocabulary well. Doc C mentions "email" and "password" but in the context of notifications. Doc B mentions "access to your registered email" but uses different vocabulary than the query, earning the lowest bi-encoder score despite being the most contextually relevant.

Step 2: Cross-encoder input construction. For each candidate, we construct the input sequence as [CLS] + query_tokens + [SEP] + document_tokens + [SEP]. For Doc A, the query tokens include "forgot," "email," and "account," while the document contains "registered email address" and "reset link." The [CLS] token will attend to all of these simultaneously.

Step 3: Joint attention and scoring. The cross-encoder processes each concatenated pair. For Doc A, the attention mechanism highlights the tension between "forgot the email" in the query and "Enter your registered email address" in the document. The document presupposes that the user knows their registered email, which directly contradicts the query's stated constraint. The [CLS] representation captures this semantic mismatch, and the scoring head assigns a low relevance score of 0.31.

For Doc B, the cross-encoder identifies the phrase "no longer have access to your registered email" as directly addressing the query's constraint. The attention heads align "forgot the email" in the query with "no longer have access to your registered email" in the document, recognizing semantic equivalence despite lexical difference. The phrase "alternative verification methods" is identified as the solution to the user's problem. The cross-encoder assigns a high score of 0.94.

For Doc C, the cross-encoder recognizes that while "email" and "password" appear, they are in the context of notification settings, not account recovery. The query's intent, recovering account access, finds no matching information in this document. The cross-encoder assigns a score of 0.12.

Step 4: Reranked results. The cross-encoder scores: Doc B (0.94), Doc A (0.31), Doc C (0.12). The reranker correctly identifies Doc B as the most relevant, despite it having the lowest bi-encoder score. This dramatic reversal illustrates how cross-encoders capture pragmatic constraints and presuppositions that bi-encoders miss. The necessary signal is not lexical overlap but the alignment of the specific condition stated in the query (forgotten email) with the specific scenario described in the document (no access to registered email).

Step 5: Interpreting the result. The ranking reversal between Doc A and Doc B illustrates the bi-encoder's weakness. Doc A scores high on the bi-encoder because it contains the exact words from the query: "reset," "password," "email." But it is a worse answer because it assumes the user knows their email. Doc B uses different words ("no longer have access") but addresses the exact situation the user is in. The cross-encoder, reading both texts together, can make this distinction. The bi-encoder, working with independent embeddings, cannot.