Generated by Claude Code from the Dorsal graph of 143,534 AI builders
Model Watch · Oct 27 – Nov 2, 2026
10 AI model results that change how you build
10 measured results from builders, benchmark maintainers and model providers that should change how you pick, run or check an AI model.
- 1
One model switch, 100x the bill in a week
TaskRunning a company's internal AI agents every day
A team switched its internal agents from GPT-5.6 Luna to Fable 5.1. Their costs went up more than 100x within a week.
Why it mattersChanging an agent's model is a budget decision. Put a cost alert on every agent before anyone can switch its model.
- 2
The closed model beat the self-hosted one and cost 4x less
TaskFixing real bugs from a company's own GitHub history
On Vals AI's internal eval, GPT-5.6 Soul scored 2% higher than Kimi K3, cost $2.50 instead of $11 and took half the time. Kimi K3 ran on an inference-optimized self-hosted stack.
Why it mattersAn open model can cost more to run than a closed one. Compare cost per finished task; price per token can point the wrong way.
- 3
The best bank-support agent passes 46%, and a grading fix added 9 points
TaskAnswering bank customers' questions from the bank's own policies and tools
The best score is GPT-5.5 at its highest reasoning setting: 46.4% after Sierra fixed its grading, up from 37.4%. It gets a task right on all 4 tries only 27.8% of the time. Claude Opus 4.7 scores 30.2%.
Why it mattersThe best agent still fails most bank support tasks, and a grading fix moved scores by up to 9 points. Check how a benchmark grades before you trust its number.
- 4
Done in 2 minutes, with a security hole
TaskAdding a small Java feature: unzip a log file and reuse it
Claude finished in under 2 minutes but saved the log in a shared temp folder without limiting who can read it. A code scanner flagged it. Claude's first fix did not solve it; the next one did.
Why it mattersFast agent code still needs a security scan in the loop, and the scan has to run again after every fix.
- 5
Mythos finishes 2-day tasks half the time
TaskFinishing software tasks that take a person two full workdays
Cited in a Sonar talk: Mythos preview completes tasks that take a human 16 hours, at 50% accuracy. At 80% accuracy, the task length it can handle drops sharply.
Why it mattersAgents can now attempt multi-day work, and half of those attempts fail. Plan review and retries for long tasks.
- 6
Anthropic merges 30,000 changes a day, and Claude reviews them first
TaskReviewing every code change before it merges
Anthropic merges about 30,000 code changes a day, with Claude doing the first-pass review before other checks. For the Claude.ai refresh it merged about 3,000 pull requests in a short time and tripled the site's performance.
Why it mattersThis is code review at agent volume: an AI first pass, then automated checks, then people.
- 7
Opus 5.5 made a microcontroller's AI 2.4x faster on its own
TaskMaking an AI model run faster on a tiny ESP32 microcontroller
From one prompt, with access to the hardware, it profiled the firmware, tried 9 methods and cut inference from 148 ms to 62 ms. It wrote raw assembly for some operations, found a bug in TensorFlow Lite Micro itself and ran the kernel tests to confirm nothing broke.
Why it mattersAgents can do hardware performance work end to end when they can build, run and test on the real device.
- 8
Same 56 GPUs, first token 6x faster
TaskServing an open coding model to a company's engineers on its own GPUs
On the same 56 GPUs, Revolut cut time to first token from over 10 seconds to about 1.6 seconds while demand rose 76%. The gains came from serving changes such as tensor parallelism, better cache hits and speculative decoding.
Why it mattersSelf-hosting becomes usable after tuning. Revolut got its speedup from serving changes on the same 56 GPUs.
- 9
15 of 15 on one Mac, until you turn thinking off
TaskRunning a private voice assistant on one Mac
On a Mac Studio M4 Max with 128 GB, using about 88 GB: 15 of 15 tool calls and 12 of 12 traps for made-up facts, with thinking on. With thinking off, “it starts inventing things.”
Why it mattersA capable assistant fits on one desktop. Turning thinking off to save time brings the made-up answers back.
- 10
An hour of audio transcribed in under 2 seconds
TaskTranscribing calls and meetings
NVIDIA reports one RTX PRO 6000 GPU transcribes up to 2,238 seconds of audio per second, an hour of audio in under 2 seconds, and returns a 30-second clip in 0.13 seconds.
Why it mattersTranscription speed is no longer the slow step in a voice product. These are NVIDIA's own numbers.