Joel acknowledges that METR's baseline methodology is imperfect and would ideally have 100x more resources (100 baselines per task, top-tier engineers, wider task distributions), but current limitations don't invalidate the core finding of exponential progress because doubling time is robust to baseline variability.
factualpending
Speaker
Joel BeckerEvidence Quote
“some of the details... aren't going to matter as much as you might naively think. So choosing the... shortest baseline time... or the longest time... it's actually not going to make that much difference”
Created: 8/12/2026, 6:03:24 PM
My Notes
Loading notes...