Evals Explained, From First Principles
文章目录
部分视频或图片暂未收录,已在正文保留提示;需要时可查看原文。查看原文 ↗
Whenever any frontier lab says its new model is better at coding, it means the model scored higher on some set of coding tasks graded in some particular way. When you change those tasks or infact, the grading, there's high chance the claim can also flip. So evals are honestly one of the most boring looking but most important parts of working with LLMs, and you should understand them properly before you trust any number on a leaderboard or ship a change based on one.
So what is an eval? It's basically a test for an AI system, you give it some input, run some grading logic on whatever comes out and you get a score. Sounds same as a unit test, but there is one big difference. A normal function gives you the same output for the same input every single time, while a language model can answer the same prompt differently on every run. OpenAI's evaluation guide says the same thing, and that's why evals behave much more like experiments than like unit tests.
In this article I"ll build up evals from the basic pieces, using what Anthropic, OpenAI and Google DeepMind have actually published about how they do it.
What is an eval actually made of?
Anthropic put out an engineering post on agent evals in January 2026, and the vocabulary in it is really clean so I'm going to borrow it. A task is one test case, with defined inputs and success criteria. A trial is one attempt at that task, and you usually run a few of them because the model output keeps changing. A grader is the logic that scores some part of the result, and one task can have more than one grader. The transcript is the full record of a trial with every message, tool call and intermediate result, and the outcome is whatever the final state of the world is when the trial ends.
The transcript vs outcome difference matters alot more than it looks. Anthropic's example is a flight booking agent. The transcript can end with the agent saying "your flight is booked", but the outcome is whether a reservation actually exists in the database or not. If your grader only reads the agent's last message, it will happily pass an agent that just lied to you.
There are two more pieces. The evaluation harness is the infra that runs the tasks, records everything and adds up the scores. The agent harness, some people call it the scaffold, is the code that lets a model act like an agent in the first place, so its tools, prompts and the loop. When you evaluate "an agent" you are always evaluating the model and its harness together, and a better harness can move a score as much as a better model can.
Who does the grading?
There are three kinds of graders and each one trades cost against nuance differently.
Code based graders are the simplest ones. They check an exact string, run unit tests, look inside a database or count tool calls. They are fast, cheap and fully reproducible so debugging them is easy. The problem is they are brittle, a correct answer written in a slightly different format can still fail.
Model based graders, which everyone calls LLM-as-judge, ask another model to score the output against a rubric, compare two answers or check a list of statements. They handle open ended answers pretty well and scale cheaply, but they are not deterministic and they have known biases. OpenAI's guide names two of them, position bias, where the judge's verdict depends on the order the answers show up in, and verbosity bias, where it just likes longer answers more. Both labs say you should check your judge against human labels before trusting it at scale. Anthropic also suggests giving the judge a way out, like letting it answer "Unknown" when it can't tell, and using a separate judge for each thing you're grading.
Human graders are the gold standard for quality and they're what you use to calibrate the other two. They are also slow and expensive, and for specialised tasks you need actual experts.
In practice good eval suites mix all three. Anthropic's advice is to use deterministic graders wherever they work, add model graders where you need judgement, and use humans carefully to keep the model graders honest.
Why should you grade the outcome and not the path?
It's tempting to write a grader that checks whether the agent took the exact steps you expected, in the order you expected. Anthropic warns against this, because agents keep finding valid routes you never thought about. In one case Claude Opus 4.5 "failed" a task from the τ2-bench flight booking benchmark because it found a loophole in the airline's policy that got the customer a better result than the intended answer.
Bad grading can also hide real ability. On CORE-Bench, which is a benchmark about reproducing scientific results, Opus 4.5 first scored 42%. Part of the reason was a grader so strict that it rejected "96.12" when the expected value was "96.124991...". After the team fixed the rigid grading, the ambiguous instructions and the tasks that couldn't be reproduced, and loosened the scaffold a bit, the same model scored 95%.
So grade the result that actually matters, and give partial credit when a task has multiple parts. A support agent that verifies the customer properly but then fails to issue the refund has still done better than one that fails at the very first step, and a good grader should be able to tell those two apart.
Why run the same task more than once?
Because the model is random, one trial tells you very little. There are two metrics people use to turn repeated trials into a number and they answer different questions.
pass@k is the chance that atleast one of k attempts succeeds. If one trial succeeds with probability p and trials are independent, it is:
[未转换内容,参见原始 JSON]
When you have actually run n samples per task and c of them passed, the standard unbiased estimate comes from OpenAI's Codex paper:
[未转换内容,参见原始 JSON]
pass^k, which you read as "pass hat k", is the chance that all k attempts succeed:
[未转换内容,参见原始 JSON]
With the same per trial success rate these two go in opposite directions. Anthropic's example uses a 75% success rate over three trials, where pass^3 is 0.75 cubed, so around 42%. With the same numbers, pass@3 comes out to around 98%.
Which one you want depends on your product. pass@k fits cases where one good answer is enough, like a coding assistant that can try a few patches and keep the one whose tests pass. pass^k fits anything customer facing, because a user who hits the one bad run out of three still had a bad experience.
How sure can you be about a score?
An eval score is a measurement so it has error bars, and most leaderboards just leave them out. Anthropic published a short statistics guide on this in late 2024, based on a paper by Evan Miller. The first recommendation is to treat the questions in an eval as a sample from a much bigger universe of possible questions, and report the standard error with every score:
[未转换内容,参见原始 JSON]
For an eval scored right or wrong, the standard deviation s is the square root of p(1 - p). Say you have an eval with 500 questions and a model scores 80%. The standard error is about 1.8 percentage points, so the 95% interval goes from roughly 76.5% to 83.5%. If another model scores 82% on the same eval, those two numbers alone don't tell you which one is actually better.
The guide adds four more ideas. When questions are grouped, like a bunch of questions about the same reading passage, they are not independent and the standard error has to be clustered by group, and Anthropic says clustered errors on popular evals can be more than three times as big as the naive ones. Averaging several samples per question, or using the model's probability of the right answer when you can, removes some of the noise. Comparing two models on the same questions with a paired test cancels out how hard each question is, which helps alot because models tend to find the same questions hard, with correlations between 0.3 and 0.7 on popular evals. And before you build an eval, a power analysis tells you how many questions you need to detect the size of difference you actually care about.
What is the difference between capability and regression evals?
Anthropic splits evals by the question they answer. A capability eval asks what the system can do well, and it should start with a low pass rate so there's room to improve. A regression eval asks whether the system still does what it used to do, and it should sit close to 100% so any drop stands out straight away.
The two are connected. Once a capability eval climbs to a high pass rate it can graduate into the regression suite and run on every change. At that point it can't tell you anymore whether you're improving, which is what people mean when they say a benchmark is saturated. SWE-bench Verified is the usual example, frontier models now score above 80% on it, so there's very little room left to separate the best of them.
What makes a good eval task?
The most practical advice in Anthropic's post is to start small and start early. Teams often wait until they can write hundreds of tasks, but 20 to 50 simple tasks taken from real failures is enough to begin, because early changes to a system have big effects that a small set can easily catch. OpenAI's guide says something similar, log everything while you develop so you can mine those logs for eval cases later.
A good task is unambiguous. Anthropic's test for this is whether two experts would independently give the same pass or fail verdict. Every task should also have a reference solution that passes all of its graders, which proves the task is solvable in the first place. If a task scores 0% across a hundred trials, most of the time the task is broken and not the model, and an unspecified file path in a Terminal-Bench task is one example Anthropic gives.
Balance matters too. If you only test cases where a behaviour should happen, you end up teaching the system to do it everywhere. When Anthropic built evals for web search in Claude.ai, they tested both questions that need a search and questions that don't, so the model learns when to search and when to just answer.
Each trial should also start from a clean environment. In one of Anthropic's runs Claude scored better than it should have because it could read the git history left over from earlier trials. OpenAI's guide adds that your dataset should cover typical cases, edge cases and adversarial ones, and should look like real traffic, not whatever was easiest to write.
Can the way you score make a model worse?
Yes it can, and OpenAI's research on hallucination is the clearest example of this. Most benchmarks score answers as right or wrong, nothing in between. Under that rule guessing is always the better strategy, because a guess sometimes gets a point while "I don't know" never does, so a model tuned only for accuracy learns to answer confidently even when it has no idea.
OpenAI showed this with numbers from the GPT-5 system card on SimpleQA, which is a set of short factual questions. The older o4-mini abstained on only 1% of questions and got 24% right, but it was wrong 75% of the time. gpt-5-thinking-mini abstained on 52%, got 22% right and was wrong only 26% of the time. On an accuracy leaderboard the older model looks slightly better, even though it gives about three times as many wrong answers.
OpenAI's fix is to change the scoring so that confident errors cost more than saying "I don't know", and to give some credit for showing uncertainty when it's appropriate.
The other way scores go wrong is contamination, where the model has already seen the test questions somewhere in its training data. Google DeepMind piloted what it calls double-blind evaluations in 2026, where outside evaluators tested a Gemini model with prompts Google never saw, and the evaluators never saw the model's weights either. That kind of setup keeps the test questions from leaking into the model's training data, but it still can't tell you whether the benchmark itself is measuring anything useful.
Are automated evals enough on their own?
No single method catches everything. Anthropic describes evals using the Swiss cheese model from safety engineering. Automated evals run before launch and on every change, and after launch you add production monitoring, A/B tests, user feedback, manual review of transcripts and the occasional careful study with real people. Every layer has holes, the idea is just that the holes rarely line up.
The one habit that ties all of this together is reading transcripts. A score tells you how often something failed, but only the transcript tells you if the agent made a real mistake or the grader rejected a perfectly valid answer. Anthropic's rule of thumb is that every failure should look fair once you read it, and if it doesn't, the problem is in your eval and not the model.
Practice it on TensorTonic
Most of the maths behind good evals is only a few lines of code. You can compute the standard deviation that goes into a standard error at tensortonic.com/problems/sample-var-std, and build a confidence interval by bootstrapping at tensortonic.com/problems/bootstrap-mean. For model based grading, tensortonic.com/problems/cs336-l12-judge-length-debiasing deals with the verbosity bias of LLM judges, tensortonic.com/problems/cs336-l12-pairwise-elo-fit turns pairwise preferences into ratings, and tensortonic.com/problems/cs336-l12-multiple-choice-likelihood scores multiple choice answers by likelihood. tensortonic.com/problems/expected-calibration-error measures whether a model's confidence actually matches how often it's right.
If you want a broader overview, the guide at tensortonic.com/guides/ml-evaluation-metrics covers the classic metrics, and tensortonic.com/ml-design/generative-evaluation goes through how evaluation works for generative systems.
p.s. - I read through these lab posts and put down my own thoughts on them, and then AI helped me with the drafting and fact checking. So if something feels off or you think I missed something, do let me know.











