How AI Benchmarks Work—and What the Scores Actually Mean

What is being tested: the model or the whole system?

Training is the stage in which a model learns patterns from data. Inference is running the trained model on an input to generate an output. Evaluation uses those outputs to test whether a specified model or system meets a scoring rule. It is not a new full training process for each test question. Keeping evaluation problems separate from training matters because prior exposure can make the test less informative about unfamiliar tasks. [2]

Some evaluations test a model answering a prompt. Others test an AI agent: a model inside software that lets it take steps using tools. A coding agent might inspect files, edit code and run commands. The surrounding software is often called a scaffold. It manages the interaction loop and determines which tools the model can use. [1]

METR, an AI evaluation research organisation, describes selecting a model-and-scaffold setup on a development set before evaluating it on a separate, larger test set. The development set helps researchers choose the setup; the test set measures the resulting system. A score from this process belongs to that combination, not automatically to every app using the same model. [1]

How a test task becomes an AI output

First, the evaluator defines the task and supplies its instructions and starting information. In a coding benchmark, that can mean a reported software issue and the repository as it existed before the fix. A repository is the collection of code and related files that the system is asked to change. [2]

Next, the system works within the permitted environment. The scaffold can pass tool results back to the model so it can decide what to do next. Time limits and limits on generated text constrain how much work the agent can attempt. METR describes independent runs of this kind, followed by checks for scoring problems. [1]

The output is the object being judged. For a software task it may be a code change, not merely a written explanation of the bug. A convincing description of how to fix an issue is therefore different from delivering a change that meets the benchmark's requirements. [2]

How an output becomes a pass—or a failure

A scoring rule turns the output into a result. In SWE-bench Verified, a software-engineering benchmark, the model receives the issue description and the earlier repository state without seeing the grading tests. It passes a problem only if all required tests pass after its code change is applied. [2]

One group of tests checks whether the reported issue is fixed. Another group contains regression tests: checks that previously working behaviour still works after the change. This prevents a fix from counting as successful merely because it solves the named bug while breaking something else. [2]

The distinction to watch is between the task's purpose and its scoring proxy. A proxy is a check used to stand in for the outcome you care about. Code tests make assessment repeatable, but they cannot be interpreted without asking whether they accurately represent the requested fix. OpenAI's audit of this benchmark shows how the two can come apart. [2]

What an overall score leaves out

When a benchmark reports a simple percentage of tasks passed, the number summarises passes among the tasks counted. Read the denominator: which tasks were included? Then read the aggregation rule: were tasks counted equally, weighted differently or attempted repeatedly? METR, for example, uses repeated runs and analyses success in relation to estimated human task duration. Its time-horizon result is not just a raw pass percentage. [1]

An aggregate does not tell you the result for every individual task. METR notes that, among tasks around a given duration, an agent can consistently succeed on some, consistently fail on others and have mixed results on the rest. The same overall result can therefore hide different patterns of success and failure. [1]

For a useful interpretation, keep the model version, scaffold, permitted tools and resource limits attached to the score. Treat the task selection and scoring method as part of the result too. These are the conditions of the test, not incidental footnotes. [1]

When the scoring test misjudges the solution

A test can be too specific about implementation. OpenAI describes a SWE-bench Verified example where the grading tests import a particular helper function even though its name was not required in the problem statement. A different implementation can address the stated issue yet fail because that helper is absent. [2]

A test can also demand work the instructions did not ask for. In another published example, the underlying code change addressed several issues, but the benchmark description covered only one. Tests checking the additional fixes could reject a solution that correctly handled the described task. [2]

OpenAI, a model developer and benchmark co-creator, reported material test-design or problem-description issues in 59.4% of 138 audited tasks. Those tasks were selected because o3 did not consistently solve them. The figure applies to that selected subset—not to all 500 SWE-bench Verified tasks, and not to every AI benchmark. It illustrates why the scorer itself needs scrutiny; it is not a universal defect rate. [2]

Why prior exposure to test answers matters

Training contamination means evaluation problems or their solutions entered the material a model learned from. If the purpose is to measure performance on unseen problems, that exposure weakens the interpretation. Public software repositories make this a relevant concern for coding tests drawn from already resolved issues. [2]

OpenAI reported evidence that the frontier models it tested had seen at least some SWE-bench problems or solutions during training. In its analysis, models could reproduce reference fixes or specific problem-statement details for certain tasks. Attribute that finding to the named audit; it does not establish that every successful answer on every benchmark is recalled from training. [2]

Contamination and faulty tests can distort results in different directions. Prior exposure can help a system pass, including by supplying hidden implementation details. A mismatched test can reject a valid solution. Neither a high score nor a failure should be interpreted without considering how the evaluation was constructed. [2]

What METR's 50% task-completion horizon means

METR's time horizon starts with human task duration: how long a human expert is estimated to need to complete each task. Researchers then fit a curve relating those durations to the AI agent's probability of success. The 50% horizon is the human task duration at which the fitted prediction reaches success half the time. [1]

It is not the number of hours the AI runs autonomously. It is not a promise of uninterrupted dependable operation, and it does not mean that every task of that duration has exactly the same success probability. The time describes the human baseline for task difficulty, while the percentage describes predicted success within the evaluation's task distribution. [1]

METR's suite primarily contains software-engineering, machine-learning and cybersecurity tasks. They are designed to be self-contained and well specified. Real work can require prior project knowledge, unclear instructions and interaction with other people. A horizon from this suite cannot establish performance across every kind of work. The source page was last updated on 8 September 2026 and says it is no longer actively updated. [1]

Common questions about AI benchmark scores

These questions help translate a reported test result into a claim with a clear scope. They do not require assuming that benchmarks are useless or that scores are universal guarantees.

Why can a high-scoring AI still give me a wrong answer?

Your request may differ from the benchmark's tasks, and the app may use different instructions or tools. Even within a tested task distribution, an aggregate success measure does not guarantee success on each task. A good score is evidence about the measured setup, not a certificate for an individual response. [1]

Do tools make a benchmark result unfair?

Not inherently. If the evaluation is intended to test an agent, its tool setup is part of what is being measured. The misleading step is presenting that system result as if the model achieved it without those tools, or comparing setups without explaining their differences. METR explicitly describes evaluating a model combined with a scaffold. [1]

What should I read beside the headline number?

Look for the exact model and configuration, the tasks included, the instructions, the permitted tools, the resource limits and the scoring rule. Also check whether the evidence concerns unseen tasks and whether a quoted problem rate comes from a selected subset. These details tell you which conclusion the evaluation can support. [1] [2]

If the deployed model is changed, its quality needs checking in that form rather than being assumed from another configuration. Quantization—representing model values at lower numerical precision—is one example of a deployment change worth understanding separately.