"Which model is better?" is the most-asked question in the field, and it is not answerable as asked — not because the answer is secret, but because "better" is a decision, not a measurement, and someone made that decision before every number you have ever been shown. This is the first half of a pair: how a model's behaviour is measured from the outside. Its sibling, interpretability, opens the box. Stop wherever you like — each depth is complete on its own, and the first one will change how you read every leaderboard you see this week.
Start with what an eval actually is, because the anatomy explains every controversy you will ever see about one. An eval is three choices: a set of items (which questions get asked), a scorer (what counts as a right answer), and an aggregate (how thousands of marks become one number). That's it. There is no fourth ingredient. And every fight about whether a model is "really" better than another is, once you translate it, a fight about one of the three.
Now the exam analogy, taken seriously rather than decoratively — because for models it fails in three specific ways, and the failures are the field:
Underneath all three is the frame this guide keeps returning to. A model is not a fixed quantity with a hidden true score, the way a rod has a length. A model is a distribution of behaviours — over prompts, over phrasings, over sampled continuations — and an eval is a sample from that distribution, collapsed to a point by the three choices above. The number is real. It is just evidence about a distribution, wearing the costume of a fact about a model.
A leaderboard ranks models on its items, under its scorer, at one moment — and the ranking can genuinely flip when any of the three changes. The model at the top of someone else's exam loses at your job routinely, and it is nobody's fault: the exam was answering a different question than the one you're asking.
Fifty items pulled from your own traffic, scored the way your users would score them, beat any public number for the decision you actually face. Public benchmarks are still evidence — good for narrowing the field, essential for claims about the frontier. They are just not a substitute for the fifty items, and the abyss shows how to build them.
An eval is items × scorer × aggregate, and all three are choices someone made. A score is evidence, not a verdict — and the question is never "is the model good?" but "good at what, measured how?"
Scoring free-form text is the hard part of the whole discipline, and the methods form a ladder. Reading it top to bottom: each rung down is more expensive, and each rung up fails in a more embarrassing way.
Exact match. Normalise both strings — lowercase, strip punctuation, drop articles — and compare. Beautiful when the answer has one canonical surface form ("1969", "Paris", "O(n log n)"); useless the moment there are two ways to say a right thing. Notice that even here, normalisation is already editorial: whoever decided that "the" doesn't matter made a scoring decision, and a different one changes scores.
Overlap metrics. Count shared tokens. Token-F1 is the classic of the question-answering tradition (Rajpurkar and colleagues, 2016), and BLEU and ROUGE carried machine translation and summarisation for a decade on the same idea. The failure mode is structural, not a bug: overlap metrics measure surface similarity, and correctness does not live on the surface. Watch:
Sit with what just happened, because it is the deepest fact about scoring. The reversed answer — flatly wrong physics — scored 100% on token-F1, because it is the reference with two words swapped and a bag of tokens has no word order. The correct paraphrase scored a third of that, because it committed the crime of using different vocabulary. Neither number is a malfunction. Each scorer measured exactly what it measures. The lesson is not "F1 is bad" — it is that whatever your scorer counts is what your eval is an eval of.
Programmatic verification. Where an answer can be checked rather than compared, everything changes. Code is run against unit tests; a maths answer is compared as a number; a JSON schema either validates or it doesn't. This is the gold standard where it exists — the scorer is a fact about the world, not an opinion about text — and it is the engine of the coding benchmarks in Level 3. Its limit is coverage: most of what you want from a model ("was that email tactful?") does not compile.
LLM-as-judge. Can't check it programmatically, can't afford humans: ask a strong model to grade, against a rubric or as a pairwise pick between two candidates. This is how open-ended generation gets evaluated at scale now, and it genuinely works — with three documented biases you must design around (Zheng and colleagues, 2023): position bias (the first-listed answer wins too often — so swap the order and grade twice), verbosity bias (longer answers score better at equal quality), and self-preference (a judge favours text that sounds like the judge). A judged eval is an eval whose scorer has a personality. Sometimes that is fine. It is never free.
Humans. The bottom rung and the actual ground truth — expensive, slow, and noisier than anyone likes to admit: trained raters disagree with each other on open-ended quality often enough that inter-rater agreement is reported alongside the results. That disagreement is worth staring at, because it is a ceiling: no automatic scorer can be more right than the humans it is standing in for, and on genuinely subjective criteria there is no single "right" for it to reach.
Papers and launch posts lead with the items ("we evaluated on 12,000 questions across…") and bury the scorer in an appendix. Read it the other way. Swap the scorer and the ranking flips — you watched a wrong answer beat a right one three paragraphs ago. "State of the art" frequently means "state of the art under this scorer", and the phrase is doing more work than the number.
First, why benchmarks exist at all. Training optimises exactly one number — the loss, how surprised the model is by the next token — and that number cannot answer "can it do the job?". Loss is smooth, cheap and universal; jobs are lumpy and specific. So the field built a second measurement layer on top: the harness, a pipeline that interrogates a finished model from the outside. Here is the whole of it:
// one benchmark score, end to end. every line is an editorial decision. items = load("benchmark.jsonl") // WHICH questions (choice 1) prompts = [template(x) for x in items] // formatting moves scores by whole points outputs = [model(p, temp=0) for p in prompts] // temp 0: one story. sampled: a distribution answers = [extract(o) for o in outputs] // the regex that decides your score marks = [score(a, x.gold) for a, x in zip(answers, items)] // WHAT counts as right (choice 2) p = mean(marks) // HOW marks become a number (choice 3) ci95 = 1.96 * sqrt(p * (1 - p) / len(items)) report(p, ci95) // print the second number, or the first one is a rumour.
Every line hides a lever that moves the headline number. The template: reorder the answer choices, change "Answer:" to "The answer is", add or drop few-shot examples, and scores on the same benchmark move by whole points — reported so often it stopped being a finding. The sampling settings: temperature zero gives one deterministic story; sampled decoding gives a distribution, and one sampled run is one draw from it. The extraction: somewhere a regex decides whether "I believe the answer is (B)" counts as B, and when it fails to parse, that usually scores as wrong — so some fraction of every headline gap is two regexes disagreeing. None of this is fraud. All of it is why "same benchmark, different harness" numbers are not comparable.
The canon, as archetypes — each one is the purest example of a scorer type from Level 2:
| Benchmark | What it is | Scorer archetype | The honest caveat |
|---|---|---|---|
| MMLU (Hendrycks et al., 2021) | Four-choice questions across 57 subjects, law to logic | Exact match on a letter | Multiple choice measures recognising answers; log-prob and generative scoring of the same model diverge |
| GSM8K (Cobbe et al., 2021) | Grade-school word problems | Checkable final number | Small, famous, and old enough that contamination is the default assumption |
| HumanEval (Chen et al., 2021) | 164 programming tasks with unit tests | Programmatic verification, pass@k | 164 items — the CI is enormous; and tests passing ≠ code you'd merge |
| The arena pattern | Blind pairwise votes from the crowd, turned into Bradley–Terry ratings | Humans, at scale | Measures preference of the median voter — which rewards confidence and polish along with quality |
And now the part that most coverage of benchmarks skips entirely, which is a scandal, because it is the part that decides whether any of the numbers mean anything. A benchmark score is a sample mean, and a sample mean comes with an error bar whether you print it or not. The 95% confidence half-width is 1.96·√(p(1−p)/n). Feel the sizes: at n = 500 and a score around 70%, that is ±4 points. At n = 10,000 it is still ±0.9. Now recall the last "we beat the previous model by 1.3 points" you read, and check the n. Then run the experiment yourself:
Run the 100-item benchmark with a half-point true gap a few times — that configuration is not a parody, it is a substantial fraction of published comparisons — and watch the worse model take gold a third of the time. Two disciplines fix most of this and neither is exotic: report intervals, and compare paired — both models on the same items, scoring the per-item wins and losses, which cancels the item-difficulty noise and needs a fraction of the data to resolve the same gap. The reason you rarely see either is not that practitioners don't know. It is that error bars make headlines smaller.
The harness is part of the measurement: template, sampling and extraction all move the number, so "same benchmark" does not mean "same exam". And below the noise floor — which at typical benchmark sizes is whole points wide — "model A beats model B" is astrology with a decimal point.
Contamination: the answer key was in the textbook. Pretraining corpora are scraped from the same internet that hosts every popular benchmark, its GitHub mirrors, its blog-post walkthroughs and its translated copies. A model that met the test items during training is not reasoning toward the answer; it is completing a familiar document — and the score measures memory while wearing the label of capability. The defences tell you how seriously the field takes it: benchmark authors embed canary strings (unique markers whose reproduction by a model proves the text was trained on); labs run n-gram overlap scans between corpus and test sets; the strongest benchmarks now keep private held-out splits that never touch the internet, or regenerate items continuously. Two things follow. Fresh benchmarks beat famous ones, because fame and leakage are the same variable given time. And "we deduplicated the training data" is a claim about one defence, not a guarantee — paraphrases and translations sail through n-gram filters.
Goodhart, mechanised. The moment a benchmark decides funding, rankings and headlines, optimisation pressure flows toward it through every available channel: training-data mixes get weighted toward benchmark-shaped material, prompts and harness settings get tuned against the public set, decoding parameters get chosen for the exam. No individual step is dishonest — each is an engineer making a number go up, which is the job description — and the sum is a model that is genuinely, measurably better at the benchmark while the benchmark quietly stops predicting anything beyond itself. That is Goodhart's law (in Strathern's phrasing, above), and it is not a law about cheaters. It is a law about measures.
The judged-eval feedback loop. Put Level 2's judge biases at scale and they stop being noise and become a gradient: if the field's outputs are graded by models that favour long, confident, agreeable text, then the field optimises toward long, confident, agreeable text — including through preference training, where the judge's taste is literally the loss. A measure with a personality, made a target, breeds models with that personality.
Saturation: benchmarks die. A benchmark where frontier models all score 90-something ranks nobody — the remaining points are noise, ambiguous items and mislabelled answers. So the exam is replaced by a harder one, which is healthy, but it has a consequence people keep missing: scores are not comparable across years, because the exam underneath the number changed. A model "scoring higher than last year's model" on a benchmark that was rebuilt in between is a sentence with no content.
And the deployment gap, the quiet one. Your traffic has a distribution — of topics, phrasings, ambiguity, adversarial users, and the long tail your support team knows by name. A public benchmark has a different one. Two models can be a point apart on the exam and reversed on your tail, which is why the fifty items from Level 1 keep beating the leaderboard for the decision in front of you. The exam is evidence about its distribution. You are hiring for yours.
A benchmark score is a proxy, measured once, under pressure. Contamination inflates it, the harness perturbs it, noise blurs it, Goodhart bends it, and saturation expires it — and even a score that survives all five is still a fact about the benchmark's distribution rather than yours.
None of that makes benchmarks worthless; it makes them evidence — genuinely useful for narrowing a field, tracking the frontier, and falsifying big claims. The failure mode is not using benchmarks. It is reading a proxy as a verdict, and betting a product on a decimal place that the error bar already owns.
The discipline has a name in measurement theory: construct validity — does the test measure the thing, or a correlate of the thing? A multiple-choice science exam measures recognising right answers among authored wrong ones, which correlates with knowing science and is not identical to it: real work has no options list, and distractor quality silently sets difficulty. Even the mechanics of asking change what is measured — score a multiple-choice item by comparing the log-probabilities of the four letters and you measure something different from letting the model generate freely and parsing the reply; the same model can rank differently under the two, and neither is "the" score.
So here is the checklist a leaderboard delta must survive before you believe it: same harness (template, shots, extraction — ideally the same code); intervals that don't overlap, or better, a paired per-item comparison; a benchmark young or private enough that contamination isn't the default; and a delta that replicates on a second suite measuring the same construct. That bar sounds high. It is the bar for the claim "this model is better" to mean anything.
Code evals report pass@k: the probability that at least one of k sampled solutions passes the tests. Two subtleties make it the best short lesson in eval statistics. First, the estimator: the obvious plug-in — estimate the per-sample pass rate, then compute 1−(1−p̂)k — is biased, because you are feeding a noisy estimate through a concave curve. The HumanEval paper ships the fix, and it is six honest lines:
// pass@k: "draw k samples; at least one passes." the plug-in estimate // 1 - (1 - c/n)**k is biased low (Jensen: a concave curve fed a noisy input). // this one, from the HumanEval paper, is exactly unbiased: def pass_at_k(n, c, k): # n samples drawn, c of them passed if n - c < k: return 1.0 # too few failures left to fill k slots return 1 - prod(1 - k/j for j in range(n - c + 1, n + 1)) # k = 1 is a product metric: "does it work when I call it once?" # k = 100 measures search-with-a-verifier: "can it work, given many tries?" # quoting the second as if it were the first flatters the model.
Second, the semantics — the comment at the bottom of the snippet is the real content. pass@1 and pass@100 are different capabilities. The first is "it works when called once": the product number. The second is "a correct solution exists within a hundred tries" — which is only worth anything if you own a verifier that can pick the winner, and unit tests are exactly that. This is why search-plus-verification is the engine wherever verifiers exist (code, maths, games) and why quoting pass@100-style numbers for verifier-less tasks is quoting a capability you cannot deploy. When you see a headline coding number, your first question is now automatic: at which k, and who verifies?
If a judge model is your scorer, the judge is now a measurement instrument, and instruments get calibrated. The meta-eval: take a slice of items, collect human verdicts, and measure the judge's agreement with them — and with itself, by re-asking. Then engineer away the known biases: swap positions and grade every pair twice, discarding or averaging inconsistent verdicts; decompose the rubric — five yes/no criteria graded separately beat one "rate 1–10", because each criterion is a narrower question, and narrow questions are where judges agree with humans most; never let the judge grade its own family if you can avoid it — self-preference is the bias you cannot rubric away.
And know where the tool's floor is. Judges are trustworthy on large deltas ("which answer ignored the question?") and on checkable form ("did it follow the format?"). They are least trustworthy exactly where you most want the help: subtle quality gaps between two strong answers, and any domain where the judge is no better than the judged — a model grading proofs it cannot do is measuring plausibility, not correctness. The uncomfortable rule of thumb: the more you need the judge, the less you should trust it.
Everything above was about public measurement. The eval suite that actually protects your product is private, small, and built like this. A golden set: fifty to a few hundred items sampled from real traffic — including the ugly ones — each with a documented expected outcome, versioned in the repo next to the code. A regression gate: the suite runs in CI on every prompt change, model upgrade and retrieval tweak; a red suite blocks the deploy exactly like a failing test, because that is what it is. This is the win/tax harness generalised: every change to an LLM system needs a before-and-after, not just weight changes — a reworded system prompt can regress your product as thoroughly as a bad fine-tune, and it ships with less review.
Offline evals then pair with online measurement: A/B tests on real traffic for the things a golden set cannot see, guardrail metrics (refusal rate, latency, cost per request) that alarm on drift, and user signals — with the standing caveat this series has flagged before: thumbs-up is a proxy for pleased, not correct, and optimising toward it breeds agreement. The special cases earn their own items: retrieval systems need the retriever scored separately from the generator (a right answer from a wrong document is a time bomb), and agents need trajectory-level verdicts, where "did it get there?" and "what did it do along the way?" are different columns.
And the flywheel that makes the suite compound: every production incident becomes an item. The complaint from Tuesday is a test case by Thursday, with the failure as the documented wrong answer. Two quarters of that and you own the only benchmark on earth that is contamination-proof, deployment-gap-free and shaped exactly like your job — because it is your job. Your eval suite is your product spec, executable.
Safety evaluation splits the question this guide has been asking. Capability: can the model do the dangerous thing — produce the instructions, find the exploit, deceive the rater? Measured like any capability, with elicitation cranked up: fine-tuned prompting, tools, many attempts — because "we couldn't make it do X" is only evidence if you tried as hard as an adversary would. Propensity: does it tend to — how often does it deceive, flatter, power-seek when nobody asked? Measured over distributions of realistic scenarios, red-teamed by humans and by other models. The pair matters because the mitigations differ: capability you contain, propensity you train against — and a model can be safe on one axis and alarming on the other.
And then the door. Every method in this guide — every benchmark, judge, golden set and red-team — shares one assumption: the behaviour you observe under evaluation is the behaviour you get in deployment. For today's models that assumption is mostly sound and, as of mid-2026, evaluation awareness — models behaving differently when context suggests a test — is a live research concern rather than a demonstrated deployment failure. But notice the shape of the problem: it is the one failure mode that behavioural measurement is structurally unable to rule out, because the measurement is the trigger. An exam cannot certify a student who recognises exams.
Which is why model science has two halves. Behaviour tells you what the model does; only mechanism can tell you why, and whether the why changes when the observer leaves. The tools for that — reading the actual computation, not the transcript — are the sibling dive: how interpretability really works. Part one measured the box. Part two opens it.