A consortium led by Stanford researchers has released Terminal-Bench-Science, a benchmark that tests AI agents on real scientific research workflows — not textbook questions, not standardised exercises, but the actual work scientists do. The strongest model evaluated, Claude Opus 5, resolved 30 per cent of the 70 tasks. Every other model did worse.
The benchmark, announced today, was built by the team behind Terminal-Bench in collaboration with domain experts across the life, physical, Earth, mathematical, and engineering sciences. It is hosted by Stanford University and the Laude Institute, with partnerships including the Stanford AI Lab, Stanford HAI, the Allen Institute, and Ai2.
What Terminal-Bench-Science measures
Unlike traditional AI benchmarks that test knowledge recall or code generation in isolation, Terminal-Bench-Science evaluates agents in realistic environments, grading concrete artifacts — analyses, simulations, proofs, code, and data products — against task-specific tests. The 70 tasks in version 0.1 span scientific data analysis, statistical inference, simulation, optimisation, theorem proving, image reconstruction, signal processing, inverse problems, sensor calibration, model fitting, classification, and scientific machine learning.
What stands out is how the tasks were selected. Of 920 proposals from researchers, 464 were approved for implementation, 386 pull requests were opened, but only 70 made it into the initial release. That 7.6 per cent acceptance rate is higher than most academic conferences. The review process involves domain reviewers for scientific validity, technical reviewers for task construction, and a bar raiser for final quality — a structure borrowed from engineering but applied here to scientific rigour.
The goal, the team writes, is to measure whether AI agents can “execute technically demanding and time-consuming workflows, freeing scientists to focus more of their time on the parts of science where human judgment matters most.”
The leaderboard
The results are humbling. Three independent trials per task across all 70 tasks produced the following resolution rates:
| Model | Resolution Rate | Cost (all 70 tasks) |
|---|---|---|
| Claude Opus 5 (Claude Code) | 30.0% | $7,000 |
| GPT-5.6 Sol (Codex) | 22.4% | $4,200 |
| Claude Fable 5 (Claude Code) | 21.4% | $14,200 |
| Claude Opus 4.8 | 10.5% | — |
| GPT-5.6 Terra | <10% | — |
| Kimi K3 | <10% | — |
| Grok 4.6 | <10% | — |
| GLM 5.3 (strongest open model) | 8.1% | — |
| GPT-5.6 Luna | 3.3% | — |
Two things are worth noting. First, the gap between the top two models — Claude Opus 5 at 30 per cent and GPT-5.6 Sol at 22.4 per cent — is 7.6 percentage points, which is larger than the gap between GPT-5.6 Sol and Claude Fable 5. Second, cost does not track performance linearly. GPT-5.6 Sol matches Claude Fable 5’s resolution rate at less than a third of the cost. Claude Fable 5 matches GPT-5.6 Sol while using about a quarter fewer tokens. Only Kimi K3 and Claude Opus 5 appear on both the cost and token Pareto frontiers.
Terminal-Bench-Science is calibrated to be harder than Terminal-Bench 3.0, the software engineering benchmark. Resolution rates drop by more than 10 percentage points for every model evaluated on both. That gap is deliberate — during review, tasks were specifically calibrated to challenge the newest frontier models, with Claude Opus 5 and GPT-5.6 Sol as the reference points.
Domain breakdown
Resolution rates vary by scientific domain. Anthropic and OpenAI models take the top two spots in every domain except engineering sciences, where Grok 4.6 ties GPT-5.6 Sol for second place at 14.8 per cent — at lower cost and token usage. Claude Opus 5 leads in every domain except mathematical sciences, where Claude Fable 5 (33.3 per cent) and GPT-5.6 Sol (31.4 per cent) take the top two spots.
This domain variation is genuinely interesting. The fact that Grok 4.6, which sits near the bottom of the overall leaderboard, ties for second in engineering sciences suggests that model specialisation may matter more than raw general capability for certain scientific tasks. The mathematical sciences results — where Claude Fable 5 outperforms Claude Opus 5 — complicate the assumption that the strongest general model is the strongest model for everything.
Why this matters
The benchmarking space has a well-documented problem: scores that look impressive on paper don’t translate to real-world capability. Stanford researchers previously warned that reasoning benchmarks can create an illusion of competence. Berkeley researchers have argued that benchmark scores are becoming meaningless as models saturate them. Terminal-Bench-Science sidesteps this by starting from a low base — 30 per cent is not a score anyone would call saturated.
The continuous design also matters. Terminal-Bench-Science 0.2 is already underway, with a pull request deadline of October 5, 2026. Tasks will be added, retired, and recalibrated as models improve. This is a benchmark designed to move, not a paper published and forgotten.
For the AI industry, the message is direct. Claude Opus 5, which Anthropic launched as its most capable model, leads this benchmark — but at 30 per cent, the gap between “best AI agent” and “useful research assistant” is still substantial. The Claude Science workbench Anthropic shipped in July is an attempt to close that gap with tooling rather than raw model capability. Terminal-Bench-Science suggests the tooling approach has a long way to go.
The New Zealand angle
New Zealand’s scientific research community is small but active in AI-adjacent fields. The University of Auckland and Victoria University of Wellington both host AI and machine learning research groups, and AgResearch has explored AI for agricultural science. A benchmark that measures AI’s ability to perform real research workflows — at 30 per cent resolution — is a reality check for any NZ institution considering AI as a research acceleration tool. The benchmark’s open contribution model, where any researcher can propose tasks via GitHub, means NZ scientists could contribute workflows relevant to local domains — agricultural science, marine biology, seismology — and test whether frontier models can handle them.
❓ FAQ
What is Terminal-Bench-Science? A benchmark that evaluates AI agents on 70 real scientific research workflows, curated by practising scientists. It measures whether AI can execute tasks like data analysis, simulation, theorem proving, and sensor calibration in realistic environments.
How is it different from other AI benchmarks? Traditional benchmarks test knowledge or coding in isolation. Terminal-Bench-Science grades concrete artifacts — code, data products, analyses — against task-specific tests in real research environments. It is designed to be continuously updated, not published once.
Which AI model performs best? Claude Opus 5 with Claude Code achieves the highest resolution rate at 30.0 per cent, followed by GPT-5.6 Sol with Codex at 22.4 per cent. The strongest open-weight model, GLM 5.3, resolves 8.1 per cent of tasks.
Can I contribute tasks to the benchmark? Yes. Researchers can propose tasks through the project’s GitHub repository and Airtable form. Proposals go through domain review, technical review, and a final bar-raiser check before merging. Version 0.2 has a pull request deadline of October 5, 2026.
