factual
SWE-bench coding went from 3% to 50% in ten months
On SWE-bench, a benchmark of real-world software engineering tasks, the state of the art went from 3-4% at the beginning of the year to roughly 50% with the updated Sonnet 3.5 ten months later, and Amodei expects ~90% within another year.
factualpending
Speaker
Dario AmodeiEvidence Quote
“in 10 months we’ve gone from 3% to 50% on this task. And I think in another year we’ll probably be at 90%.”
Created: 6/13/2026, 3:36:36 AM
My Notes
Loading notes...