How Rephrased Test Data Inflated a 13B Model’s Benchmark Scores

Can a 13-billion-parameter Llama model beat GPT-4? A 2023 LMSYS research project makes that result appear possible on several benchmarks, while passing a common contamination check. The trick is a warning about how we measure models, not evidence that the modified model is generally better than GPT-4.

The researchers compare three tests: MMLU for broad knowledge, GSM8K for math and HumanEval for coding. Their Llama-rephraser comes close to GPT-4 on MMLU and exceeds the comparison score on GSM8K; a separate Code Llama-rephraser exceeds it on HumanEval. Those figures describe the paper’s particular experiment, not a current model ranking.

A benchmark provides a collection of questions or tasks for testing a model systematically. Different tests probe different subjects and capabilities. But if information from the test set makes its way into training, a score may reflect exposure to the test rather than generalization to genuinely new questions.

That leakage is called contamination. The challenge is finding it in enormous training datasets. In the researchers’ demonstration, they deliberately trained on reworded versions of benchmark items and then tested on the original items. The resulting scores show how an apparently impressive benchmark result can be inflated by exposure.

People already treat benchmark announcements with caution, and I also run my own tests based on practical use cases. Even a private test suite needs care: once its questions circulate, future models could encounter related examples during training.

A common check searches for overlapping n-grams: sequences of words shared between training and test data. String matching is relatively inexpensive, but paraphrasing or translating a question can preserve its meaning without preserving the exact wording. A clean result from that check therefore does not prove that the test is semantically independent.

The paper demonstrates how training on those variants can make a model overfit the benchmark: it performs extremely well on the specific items while the result tells us less about fresh tasks. The researchers propose a stronger LLM-based detector and investigate several public training datasets for rephrased overlaps.

For example, the authors report possible HumanEval overlaps in subsets of RedPajama and StarCoder data, and recommend fresh, one-time exams as another safeguard. These are detector-based estimates of overlapping benchmark items in the datasets examined; they do not by themselves establish how much any particular model’s score was inflated.

I also point to the paper’s discussion of reported overlap in Llama 2 and GPT-4 evaluations. Such claims need their original definitions and context: a detected overlap is not proof that a model memorized an answer, and it is not a reason to assume every result on a benchmark is invalid.

The other common approach uses embeddings to retrieve semantically similar examples. That can find relationships beyond exact strings, but setting a similarity threshold involves a tradeoff: one that catches more true matches can also flag unrelated questions from the same subject. Synthetic data generated by language models can introduce reworded material that makes the search harder.

The paper discusses Phi-1 as an example of synthetic coding data containing samples similar to HumanEval problems that an n-gram check did not detect. This matters as generated data becomes more common: its origin and relationship to evaluation material are not always obvious.

The rephrasing example is straightforward. A multiple-choice question about an 1840s reform movement is rewritten with different wording but the same meaning and answer. Training on that variant can help the model answer the original, even when simple overlap detection misses the relationship.

Contamination is not necessarily intentional. Large datasets can acquire overlapping material by accident, and an apparent match deserves examination before anyone infers misconduct. Rephrasing or translating a test question with a language model can also deliberately demonstrate the weakness of the detector.

A translation into Chinese may evade an English-oriented similarity model as well as an exact text check. Multilingual embeddings can improve cross-language detection, but their performance depends on the model and the examples. The researchers also call for fresh exams that can be used before their questions become part of subsequent training data.

Two other approaches are decoding matching and influence analysis. In decoding matching, a model is prompted to complete part of a test question; a successful completion alone does not establish that it saw that exact example. Influence methods estimate how individual training examples affect an output, but the practical calculations are approximations rather than a simple proof of exposure.

For code, the researchers consider changes to names and coding style as well as translation between programming languages. The implementation can look different while expressing the same task. Similarity models trained for multiple languages may help, although no detector catches every variation.

False positives matter too. Two multiple-choice questions can share the same arrangement of answer options while asking different things. If a filter treats every superficial match as contamination, it may remove useful training data that was never a version of the test question.

The paper’s conceptual diagram contrasts what n-gram overlap, embedding search and the proposed detector can catch. Embedding search can retrieve a broad candidate set that includes irrelevant material; running such searches across large datasets also has a computational cost. The diagram is illustrative, not a guarantee that any method captures all contamination.

In the reported experiments, models trained on English rephrasings or translated variants achieved sharply higher scores on the corresponding original test sets. The paper reports an MMLU score of 85.9 for a 13B model in this setup. The improvement demonstrates a test-leakage problem, not a broad improvement on unseen tasks.

The proposed LLM decontaminator has two stages. For each test item, an embedding search retrieves the top-k most similar training examples; a language model such as GPT-4 then judges whether each candidate is a rephrasing. This combines retrieval with a more contextual comparison, though the result still depends on both stages.

The paper compares methods using F1 scores, which combine precision and recall, rather than simply reporting the percentage of every example classified correctly. On the constructed MMLU pairs, 10-gram matching fails on the rephrased examples, while embedding and LLM-based methods vary by subject and language. The LLM detector performs strongly across the displayed cases, but some embedding results are also strong.

The table illustrates why language choice matters. In the algebra example, the non-multilingual embedding method performs well on English rephrasings but much less well on Chinese translations; the multilingual method performs better on that translation. Results differ for sociology and US history, so no single threshold or representation should be assumed to work equally well everywhere.

On HumanEval coding examples, the displayed comparison likewise shows the LLM detector doing well on rephrased Python, C and JavaScript samples, while the tested embedding methods drop on some translated code. These are results on the paper’s test pairs and do not prove perfect detection on arbitrary datasets.

The dataset table estimates that 31 of 164 HumanEval items had rephrased matches in the examined Stack subset, 26 in the StarCoder-Data subset and 21 in CodeAlpaca—18.9%, 15.9% and 12.8%, respectively. Other rows use different benchmarks and denominators, so their percentages should not be compared as a universal contamination rate for all training data.

The researchers also report lower detected overlap for the FLAN CoT and WizardLM examples against MMLU. These are findings of a particular detector on particular dataset samples, with possible missed matches and false positives. As synthetic training data grows, subtle overlap remains a reason to document datasets and evaluate with fresh, diverse tests.

My takeaway is to be skeptical of a benchmark score without understanding the training data and evaluation process. I also wonder whether my own test questions could eventually be encountered by new models, and whether I should refresh them. Multiple independent tests can help readers interpret results more carefully than a single leaderboard number.