How benchmark leakage and information contamination undermine LLMs analysis

“Our new LLM beats GPT in each benchmark!”

It’s turning into more and more widespread to listen to daring claims like this, because the hype round LLMs is large. There are new fashions each week, and at present everyone seems to be making an attempt to compete with GPT-4, which remains to be probably the most highly effective LLM.

Benchmarking is a vital a part of evaluating progress in massive language fashions.

Benchmarks like MMLU and HellaSwag are the usual for assessing language fashions on abilities like reasoning and comprehension. The scores present a snapshot of progress, with new state-of-the-art outcomes heralded as breakthroughs. LLMs are normally evaluated in a zero-shot setting, with out specific coaching on the check set, to gauge their normal skills.

This text exhibits how straightforward it’s to govern benchmark outcomes and gives strategies to take care of analysis integrity.

The Bother with Benchmarks

Typically, benchmarks don’t mirror usefulness in real-life situations. Google’s latest mannequin, Gemini Extremely, scores 90.04% on MMLU. Whereas that is a powerful rating, taking a better take a look at the analysis methodology, it’s CoT@32 (chain of thought with 32 samples). It means we’ve to immediate 32 instances to get 90% accuracy! Most of us predict an correct reply within the first attempt, particularly when interacting with a chatbot.

Sadly, this problem is simply the tip of the iceberg of LLMs analysis.

In machine studying, fashions are generally evaluated by measuring their efficiency on a check set that was not used throughout coaching. Usually, this course of permits for an unbiased estimate of how the mannequin will generalize to new information.

Benchmark leakage and information contamination are two phrases that each consult with a regarding problem: when the check information by some means leaks into the pretraining information of LLMs, resulting in inflated efficiency. It makes comparisons between LLMs unfair and…