Benchmark Contamination › Detection Methodologies

Detecting contamination requires distinguishing between real understanding and data leakage. Researchers have developed several complementary approaches, each with different sensitivity and computational tradeoffs. No single method suffices for complete detection, so practitioners typically combine multiple techniques to build confidence in their evaluations.

The most straightforward detection method uses n-gram matching. Recall from N-grams that an n-gram is a contiguous sequence of $n$ tokens. If a 13-gram (13 consecutive tokens) from a benchmark example appears in the training data, we have strong evidence of contamination. The length matters critically: shorter n-grams appear frequently by chance, while longer matches indicate deliberate inclusion or near-verbatim copying.

To see why length matters, consider that the phrase "the" is a 1-gram that appears in effect everywhere. The phrase "the capital of" is a 3-gram that appears in many geography texts. But "The capital of France is Paris, known for" at 9 words is specific enough that its presence in training data strongly implies the exact benchmark sentence was seen. At 13 tokens, accidental occurrence becomes vanishingly unlikely.

The mathematical formulation is simple but powerful. For a benchmark example $B$ consisting of tokens $[b_1, b_2, ..., b_m]$, we extract all n-grams $N_B = \{(b_i, b_{i+1}, ..., b_{i+n-1}) : 1 \leq i \leq m-n+1\}$. For the training corpus $T$, we similarly extract $N_T$. The contamination signal is:

$$C(B, T) = \frac{|N_B \cap N_T|}{|N_B|}$$

where:

• $B$: a benchmark example consisting of tokens $[b_1, b_2, ..., b_m]$
• $T$: the training corpus
• $N_B$: the set of all n-grams extracted from benchmark example $B$
• $N_T$: the set of all n-grams extracted from training corpus $T$
• $|N_B \cap N_T|$: the number of n-grams common to both the benchmark and training sets
• $|N_B|$: the total number of n-grams in the benchmark example

This ratio measures what fraction of the benchmark's n-grams appear in the training data. When $C(B, T) = 1.0$, every n-gram from the benchmark appears in training, showing near-certain contamination. In practice, thresholds around $C > 0.8$ for $n \geq 13$ give reliable detection. However, selecting the appropriate threshold requires balancing sensitivity against specificity. Too low a threshold captures coincidental overlaps; too high a threshold misses partial contamination.

The GPT-3 paper applied n-gram analysis systematically before publishing benchmark results, marking any evaluation set that had more than 13-gram overlap with the training data. This became an early standard for contamination disclosure, though it only catches verbatim copying. Papers like the OPT model card and the PaLM technical report extended this practice, including contamination analysis sections as standard procedure.

N-gram matching has clear limitations. It fails to detect paraphrased contamination and struggles with code benchmarks where variable names may differ while logic remains identical. It also produces false positives for short common phrases. For instance, "the quick brown fox" as a 4-gram might appear in both training and test by chance, even though the surrounding contexts differ completely. The method works best as a first-pass filter: computationally cheap, highly specific at large $n$, but not sensitive to semantic restatements.

For internet-scale training corpora, exhaustively checking every n-gram against every benchmark example is computationally prohibitive. A corpus of one trillion tokens contains billions of n-grams, making brute force comparison infeasible. MinHash offers an efficient approximation using locality-sensitive hashing (LSH), letting us to estimate similarity without explicit enumeration.

The key insight behind MinHash is that we can estimate the Jaccard similarity of two sets without computing the sets' intersection directly. Instead of comparing every n-gram, we compute compact "signatures" for each document, where the probability of two signatures matching equals the Jaccard similarity of the original sets. This compresses the comparison from checking billions of n-gram pairs to comparing short fixed-length vectors.

The MinHash algorithm works by:

1. Generating $k$ independent hash functions $h_1, h_2, ..., h_k$
2. For each document, computing the minimum hash value across all n-grams for each function
3. Creating a signature vector of these $k$ minimum values

The key property is:

$$P[\text{MinHash}(d_1) = \text{MinHash}(d_2)] = \frac{|d_1 \cap d_2|}{|d_1 \cup d_2|}$$

where:

• $d_1, d_2$: two documents being compared (represented as sets of n-grams)
• $|d_1 \cap d_2|$: the number of n-grams common to both documents
• $|d_1 \cup d_2|$: the total number of unique n-grams across both documents
• $\frac{|d_1 \cap d_2|}{|d_1 \cup d_2|}$: the Jaccard similarity, ranging from 0 (no overlap) to 1 (identical documents)

This equality holds because the minimum hash value under a random permutation of the universe is equally likely to come from any element. The probability that the minimum is the same element in both sets equals the probability that element is in the intersection divided by the probability it is in the union, which is exactly the Jaccard similarity.

By comparing MinHash signatures instead of raw n-grams, we reduce the complexity from $O(N \cdot M)$ to $O(N + M)$ where $N$ and $M$ are the sizes of the training and evaluation sets. This reduction makes large-scale contamination screening feasible. Locality-sensitive hashing further accelerates the process by bucketing similar signatures together, so we only compare documents that are already likely to be similar. In practice, this brings the time to screen a trillion-token corpus down from weeks to hours.

Semantic similarity detection moves beyond surface-level matching to catch paraphrased contamination. Using embeddings from models like those discussed in our earlier chapter on embedding architectures, we can detect when training and evaluation examples share meaning despite lexical differences. This approach addresses the limitation of n-gram methods by capturing semantic equivalence even when wording changes completely.

Given embedding functions that map text to dense vectors, we compute cosine similarity:

$$\text{sim}(e_{\text{train}}, e_{\text{eval}}) = \frac{e_{\text{train}} \cdot e_{\text{eval}}}{\|e_{\text{train}}\| \cdot \|e_{\text{eval}}\|}$$

where:

• $e_{\text{train}}$: the embedding vector of a training document
• $e_{\text{eval}}$: the embedding vector of an evaluation (benchmark) example
• $e_{\text{train}} \cdot e_{\text{eval}}$: the dot product measuring directional alignment
• $\|e_{\text{train}}\|, \|e_{\text{eval}}\|$: the L2 norms (magnitudes) of the respective vectors

Cosine similarity measures the angle between two vectors in high-dimensional space, independent of their magnitude. Values near 1 indicate the texts point in the same semantic direction, suggesting paraphrases or near-duplicates even when they use different words. Values near 0 indicate orthogonal (unrelated) meaning.

High similarity scores above 0.9 suggest potential contamination, though this method requires careful threshold tuning to avoid false positives from semantically similar but distinct examples. Two biology textbooks might both explain photosynthesis with high embedding similarity, yet stand for independent sources of valid training data rather than contamination. The key is to combine semantic similarity with additional evidence: if a document is both semantically similar and structurally resembles a multiple-choice question format, contamination becomes more likely.

The cost of embedding-based detection is significant. Computing embeddings for billions of training documents requires substantial GPU resources, and nearest-neighbor search over billion-scale vector databases requires approximate methods like FAISS. This makes embedding-based detection practical for targeted analysis of suspected contaminated domains, rather than exhaustive corpus-wide screening.

Building on our understanding of perplexity as a language model evaluation metric, we can detect contamination by measuring how "surprised" a model is by benchmark examples. A contaminated model will assign materially lower perplexity (higher probability) to test examples than an uncontaminated model of similar size. This method proves particularly useful when we lack access to the original training data but can compare against a reference model.

For a benchmark example $x = [x_1, x_2, ..., x_n]$, we compare the per-token perplexity:

$$PPL(x) = \exp\left(-\frac{1}{n}\sum_{i=1}^{n}\log P(x_i | x_{<i})\right)$$

where:

• $x$: a benchmark example consisting of $n$ tokens $[x_1, x_2, ..., x_n]$
• $x_i$: the $i$-th token in the sequence
• $x_{<i}$: all tokens preceding position $i$ (the context)
• $P(x_i | x_{<i})$: the probability the model assigns to token $x_i$ given previous tokens
• $n$: the total number of tokens in the example

Perplexity measures how surprised the model is by the text, geometrically averaging the inverse probabilities assigned to each token. A contaminated model assigns higher probabilities (lower perplexity) to memorized examples because it has seen them before, whereas an uncontaminated model treats them as novel and assigns more diffuse probabilities.

If $PPL_{\text{model}}(x) \ll PPL_{\text{reference}}(x)$ where the reference is an uncontaminated model of similar scale, contamination is likely. This method is particularly effective for detecting partial contamination where only fragments of examples appeared in training, as even fragmentary memorization reduces perplexity noticeably.

The main challenge with perplexity-based detection is finding a trustworthy reference model. The reference must have similar architecture and training scale as the suspect model, but without access to the potentially contaminated data. In practice, researchers have used earlier model versions trained on curated clean data, or models from different organizations whose training data is known not to include the benchmark.