The published record

Benchmarks

Every measured run, newest first. Each links to its model card, findings, and report.

Does it work every time? Opus 4.8 vs Haiku 4.5

Five runs per task: the cheapest model wins on dependability; pass@1 hides coin flips.

2026-07-07 · 15 evals · 135 runs · reliability study

Fable 5 vs Opus 4.8 on a frontier-hard tier

A four-way pass@1 tie; effort helps Opus and backfires for Fable.

2026-07-04 · 15 evals · 60 runs · frontier-hard tier

Fable 5 vs Opus 4.8 vs GPT-5.5

Four configurations, fifteen evals, each misses a different task.

2026-07-02 · 15 evals · 60 runs · corrected costs

Fable 5 low vs Sonnet 5 high vs GLM-5.2

Accuracy converges; efficiency spans an order of magnitude.

2026-07-01 · 10 tasks · 30 runs

Reasoning-effort sweep: Sonnet 5 vs Opus 4.8

Does more thinking effort pay off? Cost versus correctness across low, medium, high.

2026-06-30 · 52 tasks · 936 runs

Real bugs, both solved: the gap is cost

The pilot that established cost as the live axis once correctness saturates.

2026-07-01 · 4 tasks · 24 runs