Home / Fundamentals / Evaluation

EvaluationSeniorQuestion 2 of 6

How does benchmark contamination happen, and how would you detect and prevent it?

30-second answer

Test items leak into training through web copies, forum solutions, synthetic data from models that saw them, or tuning on test sets, inflating scores. With access to the corpus, search for long n-gram overlaps and canary strings. Without it, look for membership signals (abnormally low perplexity or high min-k% token probabilities, verbatim completion from a prefix) and, most convincingly, a performance gap on freshly written or perturbed versions of the items. Prevent it by decontaminating all training stages against reported evals, keeping private and time-split held-out sets, and refreshing benchmarks.

Try answering out loud before reading on. The walkthrough below is paced for a 40-minute interview round.

0–5 min Clarify and frame

Ask: are we evaluating our own model (with training data access) or a third-party model? Framing: "I'll cover how leakage happens, detection with and without the training corpus, the strongest evidence, and prevention."

5–15 min Core explanation

How it happens:

  • Benchmarks and their solutions copied onto web pages, forums and repositories that end up in crawls.
  • Synthetic training data generated by models that memorized benchmarks.
  • Fine-tuning datasets that include test items (sometimes in rephrased form).
  • Iterating on a test set (prompt or checkpoint selection) until the score rises: "overfitting by selection".

Why it matters: inflated scores mislead model selection, research conclusions and users.

15–25 min Deep dive: detection

With the training corpus:

  • N-gram overlap: index long n-grams (e.g. 13-grams, after normalization) of test inputs and answers; flag training documents with matches. Fuzzy matching catches paraphrase-lite copies.
  • Canary strings: some benchmarks include unique strings; search for them.
  • Embedding similarity for paraphrased leaks (higher recall, more false positives).

Without the corpus (black-box):

  • Verbatim completion: prompt with the first part of a test item; exact continuation of the rest (or of the answer options) suggests memorization.
  • Membership inference: perplexity on test items unusually low relative to similar fresh items; min-k% probability (average log-probability of the least likely tokens) is higher for seen text.
  • Perturbation and fresh-set gaps: write a parallel set with the same difficulty (new problems, different numbers, reordered options) and compare. A model that drops much more than others on the fresh set likely benefited from contamination. This is the most decision-relevant evidence.
  • Time splits: evaluate on problems published after the training cutoff (e.g. recent contest problems); compare with older ones.

25–32 min Prevention and reporting

  • Decontaminate every training stage (pre-training, SFT, preference and RL data, synthetic data) against all reported evals, and log what was removed.
  • Keep private held-out sets never released publicly.
  • Use fresh or dynamic benchmarks and time-split evaluations.
  • Separate dev and test sets for prompt and checkpoint selection.
  • Report contamination analysis alongside results (e.g. scores on clean vs flagged subsets).

32–37 min Production example

A team's new model jumped 9 points on a public grade-school math benchmark. N-gram analysis found ~4% of items with near-copies in a newly added synthetic SFT dataset. On a freshly written set of 1,000 comparable problems, the gain was only 3 points. They removed the leaked items, retrained, and reported the fresh-set result.

37–40 min Likely follow-ups

  • Is training on benchmark-like data contamination? Training on the same distribution (similar style) is legitimate; training on the items or close paraphrases isn't.
  • Can you decontaminate code benchmarks? Harder: function signatures and common solutions appear naturally; use tests on new problems.
  • How do you handle evaluating closed models? Fresh sets and time splits are the main tools.

What interviewers listen for

  • Multiple leakage paths, including selection overfitting.
  • White-box and black-box detection.
  • Fresh-set gaps as the key evidence.

Red flags

  • "We didn't train on the test set" with no checks.
  • Only exact-match deduplication.