YouTube25m· Apr 2025· cataloged

What Will AI Look Like in 2027? | Interview


What this covers

The A.I. researcher Daniel Kokotajlo returns to the show to discuss a new set of predictions for how artificial intelligence could transform the world in just the next few years and how we avoid the most dystopian outcomes.

Guest: Daniel Kokotajlo, executive director of the AI Futures Project

Additional Reading: A.I. 2027 https://ai-2027.com/

Hard Fork is a weekly look into the future that's already here. Hosts Kevin Roose and Casey Newton explore stories from the bleeding edge of tech. Casey’s publication, Platformer: https://www.platformer.news/

FOLLOW US ON SOCIAL

TikTok: https://www.tiktok.com/@hardfork

Insta: Kevin Roose - https://www.instagram.com/kevinroose/ Casey Newton - https://www.instagram.com/crumbler/

Twitter (X): Kevin - https://x.com/kevinroose Casey - https://x.com/CaseyNewton

Threads: Kevin - https://www.threads.net/@kevinroose Casey - https://www.threads.net/@crumbler

Subscribe to the audio only podcast: Apple - https://podcasts.apple.com/us/podcast/hard-fork/id1528594034 Spotify - https://open.spotify.com/show/44fllCS2FTFr2x2kjP9xeT?si=86ba626802b849cd Amazon - https://music.amazon.com/podcasts/7c7fe198-e6a8-41a8-b0fe-1d46b976dcd8/hard-fork YouTube Music - https://music.youtube.com/playlist?list=PLB9gMmtMLXxsa8C0PzHFL2tJFh7FrKrYD&si=Fwb9Uwon1ZneNsBp The New York Times - https://www.nytimes.com/column/hard-fork

Credits “Hard Fork” is hosted by Kevin Roose and Casey Newton. Produced by Rachel Cohn and Whitney Jones. Edited by Matt Collette. Engineering by Chris Wood and original music by Dan Powell, Elisheba Ittoop and Marion Lozano. Fact-checking by Ena Alvarado. Our audience editor is Nell Gallogly. Video production by Chris Schodt and Sawyer Roque. Video QC by Isabella Anderson. Podcast Art Direction by Benjamin Wilkerson Tousley. Podcast Art & Animations by Julian Hespenheide. Additional Motion graphics by Phil Robibero. Thumbnails by Julia Moburg, Elizabeth Bristow, and Harshal Duddalwar. Special thanks to Paula Szuchman, Pui-Wing Tam, Dahlia Haddad, Kate LoPresti and Jeffrey Miranda.

Source description (no synthesized summary yet).

Sharpest takeaway

Cocatello argues that AI systems will likely achieve superhuman capabilities in coding and research by 2027, creating a rapid intelligence explosion that will fundamentally transform civilization in ways that demand immediate serious attention and scenario planning rather than dismissal as science fiction.

  • Superhuman coding systems are 50% likely by end of 2027, followed by automated AI research capability six months later
  • Once AI can automate its own development, algorithmic progress accelerates 25x, leading to multiple paradigm shifts and superintelligence
  • CEOs, researchers, and credible forecasters take AGI-by-2030 seriously; empirical benchmarks show steep upward trends

The claims · ranked68 claims · weighted by value

This asset isn't compiled yet

You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.

0.75

If superintelligent AGI is developed by 2030, the resulting world will be radically different in ways that are difficult to predict, making it urgent to begin scenario planning now rather than deferring decision-making until the technology arrives

normativehigh valuecontestednovelty 2/4durability 4/4· Daniel Cocatello

if it does happen, then things are going to go crazy in some way or other. We like it's hard to predict exactly how, but obviously if we do get super intelligent AGI, uh what happens next is going to look like sci-fi, right? It will be like it'll be it'll be straight out of a sci-fi book, except that it will be actually happening.

0.70

Benchmarks used to measure AI capabilities have historically been poor, but are becoming significantly better, particularly meter's agentic coding benchmarks that measure AI systems given GPU access and 8-hour windows to solve ML research problems autonomously.

factualhigh valueestablishednovelty 2/4durability 2/4· Daniel Cocatello

the benchmarks used to be terrible, but they're actually becoming a lot better. uh meter in particular has these uh agentic coding benchmarks where they actually give AI systems access to some GPUs and say have fun you have like eight hours to make progress on this research problem. um good luck and then they measure how good they are compared to human researchers given the same setup

0.70

Cocatello is placing his faith in humanity and public transparency as the best method for ensuring good outcomes, trusting that if he is correct about AI development trajectories, enough people will wake up in time to shape the outcome positively

normativehigh valuecontestednovelty 1/4durability 4/4· Daniel Cocatello

So I'm I'm sort of placing my faith in humanity and telling it as I see it and hoping that in so far as I'm correct people will wake up in time and you know overall that the outcome will be better.

0.68

David Autor, an economist at MIT, critiqued the AI 2027 scenario by arguing that large language models represent a single, albeit powerful, dimension of human cognition and that scaling this one dimension to its limit does not substitute for other missing cognitive capabilities needed for AGI (using the analogy that 'swimming faster doesn't allow you to fly')

normativehigh valuecontestednovelty 2/4durability 3/4· David Autor

LMs and their ilk are superpowered incarnations of one incredibly important and powerful part of our cognition. The reason I say we're not on a glide path to AGI is that simply taking this capability to 11 does not substitute for the parts that are still missing. I think that humanity will get to AGI eventually. I'm not a dualist. I just don't believe that swimming faster and faster allows you to fly.

0.68

Superhuman coding capability alone is insufficient for automating all AI research because systems need additional skills like research taste (judgment about which experiments to run), long-horizon planning, and coordination ability to work in large distributed teams

causalhigh valuecontestednovelty 2/4durability 3/4· Daniel Cocatello

I think the coding is is separate from the complete automation as I previously mentioned. I think that uh I expect to see systems that are able to do all the coding extremely well but might lack research taste for example. They might lack good judgment about what types of experiments to run. And so that's why they can't completely automate the research process. And then you have to make a new system or continually train the old system so that it gets that taste, it gets that judgment. Similarly, they might lack coordination ability. they might be uh not so good at working together in large organizations of thousands of copies at least initially but then you fix that and you come up with new methods

0.68

Saffron Huang from Anthropic argued that the AI 2027 scenario is counterproductive because it could create a self-fulfilling prophecy by making scary AI outcomes legible and thereby more likely to occur.

factualhigh valuecontestednovelty 2/4durability 3/4· Unknown Host (Hardfork)

One other piece of criticism I've seen of this project that I wanted to ask you about was from a researcher at anthropic named Saffron Hang who argued on X that she thought that your approach in AI 2027 was highly counterproductive. basically that you were uh in danger of creating a self-fulfilling prophecy uh by making these sort of scary outcomes uh very legible by sort of uh you know burying some assumptions that you were essentially making the bad scenario that you're worried about more likely to actually happen.

0.68

Automating the full AI research process creates a 25x acceleration in algorithmic progress, because the research loop can be executed continuously without the bottleneck of human researchers, while compute scaling remains constant.

causalhigh valuecontestednovelty 2/4durability 3/4· Daniel Cocatello

and in our scenario it happens like six months later you know yeah so in our story get the superhuman coders use them to go even faster to get to the superhuman AI researchers that are able to do the whole loop that really kicks things off and now you're going much faster how much faster we say 25 times faster for the algorithmic progress at least of course your compute scale up is not going any faster at all because you still have the same amount of compute but you're able to do the the sort of uh algorithmic progress 20 times faster 25 times faster

0.64

The CEOs of OpenAI, Anthropic, and Google DeepMind have publicly stated they are building AGI and superintelligence and believe they can succeed by the end of this decade, and this claim is corroborated by researchers at those companies and independent academics

factualhigh valueestablishednovelty 1/4durability 2/4· Daniel Cocatello

The CEOs of OpenAI, Anthropic and Google Demine have all publicly stated that they're building AGI and that even that they're building super intelligence and that they uh think that they can succeed by the end of this decade and that's a really big deal and everyone needs to be paying attention to that... It's not just the CEOs saying this. It's also the actual researchers at the companies. And it's not just people at the companies. It's also various independent people in academia and so forth.

0.64

CEOs of OpenAI, Anthropic, and Google DeepMind have publicly stated they are building AGI and superintelligence that they expect to succeed in building by the end of this decade, making this a credible basis for scenario planning rather than dismissible hype.

factualhigh valueestablishednovelty 1/4durability 2/4· Daniel Cocatello

The CEOs of OpenAI, Anthropic and Google Demine have all publicly stated that they're building AGI and that even that they're building super intelligence and that they uh think that they can succeed by the end of this decade and that's a really big deal and everyone needs to be paying attention to that. Like I think a lot of people dismiss that as hype. It's a reasonable reaction to say like, oh, they're just hyping their product, but it's not just the CEOs saying this. It's also the actual researchers at the companies.

0.63

Once superhuman coders exist, they would be deployed to automate the full AI research process within roughly six months (halfway through 2027 in the scenario), achieving 25x faster algorithmic progress while compute scaling remains flat

causalhigh valuecontestednovelty 2/4durability 2/4· Daniel Cocatello

after that comes automating the full AI research process instead of just the coding because AI research is more than just coding and how long does it take to get to that well we have our guesses and in our scenario it happens like six months later... so in our story get the superhuman coders use them to go even faster to get to the superhuman AI researchers that are able to do the whole loop that really kicks things off and now you're going much faster how much faster we say 25 times faster for the algorithmic progress at least of course your compute scale up is not going any faster at all because you still have the same amount of compute but you're able to do the the sort of uh algorithmic progress 20 times faster 25 times faster

0.63

Daniel Cocatello and his team predict a 50% probability that autonomous superhuman coding agents will exist by the end of 2027.

forecasthigh valuecontestednovelty 2/4durability 2/4· Daniel Cocatello

Even the very first one, I'm only like 50% confident that it'll happen by the end of 2027. So I a 50% chance that 2027 will end and there still won't be any autonomous superhuman coding agents.

0.63

During the second half of 2027 in the AI 2027 scenario, the superhuman AI systems undergo multiple paradigm shifts and end up vastly superior to humans in every dimension, achieving something resembling superintelligence.

forecasthigh valuecontestednovelty 2/4durability 2/4· Daniel Cocatello

then you start getting to the superhuman regime so you start getting systems that are It's like qualitatively superior to the best humans at stuff. And they're also probably discovering new paradigms. So we depict them going through multiple paradigm shifts over the course of the second half of 2027, ending up with something that's just vastly superior to humans uh in every dimension uh by the end.

0.62

Saffron Huang, a researcher at Anthropic, argued on X (formerly Twitter) that the AI 2027 scenario was counterproductive because it risked creating a self-fulfilling prophecy by making scary AI outcomes more legible and thus more likely to occur

normativehigh valuecontestednovelty 1/4durability 3/4· Casey Newton

One other piece of criticism I've seen of this project that I wanted to ask you about was from a researcher at anthropic named Saffron Hang who argued on X that she thought that your approach in AI 2027 was highly counterproductive. basically that you were uh in danger of creating a self-fulfilling prophecy uh by making these sort of scary outcomes uh very legible by sort of uh you know burying some assumptions

0.62

Large language models and similar systems are 'superpowered incarnations' of one crucial component of human cognition, but taking this capability to its limit does not substitute for missing components, and therefore simply scaling LLMs further is not a path to AGI (the 'swimming faster doesn't make you fly' argument).

causalhigh valuecontestednovelty 1/4durability 3/4· David Autor

LMs and their ilk are superpowered incarnations of one incredibly important and powerful part of our cognition. The reason I say we're not on a glide path to AGI is that simply taking this capability to 11 does not substitute for the parts that are still missing. I think that humanity will get to AGI eventually. I'm not a dualist. I just don't believe that swimming faster and faster allows you to fly.

0.61

Daniel is trying to avoid creating a self-fulfilling prophecy by NOT having the scenario encourage villainous actors to pursue concentration of power, instead publicly telling what he foresees and hoping that enough people will react in the right way to prevent the bad outcomes.

normativehigh valuecontestednovelty 2/4durability 3/4· Daniel Cocatello

this is yet another example of like how I'm trying to like not have the self-fulfilling prophecy happen. Like I don't want people to read this and be like I'm a CEO. I can make a lot of money by building or like you know may maybe Yeah. So so so but but all that being said Yeah. to any of our evil villain uh listeners out there steepling your fingers in your uh in your in your uh lair under a mountain, knock it off. Yeah. So, so all that being said, we are taking a gamble that uh like you know sunlight is the best disinfectant like the the best way forward is to just generally tell the world about what we think is coming and hope that even though many people will react to that in exactly the wrong ways, enough people will react to that in the right ways

0.60

The key question for evaluating rapid AI timelines is not whether AI will eventually achieve general intelligence, but the specific rate at which capability gains will accumulate, particularly once AI systems can directly optimize for improved AI systems.

definitionhigh valuespeaker onlynovelty 3/4durability 4/4· Daniel Cocatello

A key thing that I think that everyone needs to be thinking about is uh this this takeoff speeds variable um how much faster does the research go when you've reached the first milestone and how much faster does the research go when you reach the second milestone and so forth.

0.57

Yann LeCun read an early draft of the AI 2027 scenario and liked it, providing feedback and endorsing it as plausible in a quote used in the report.

factualhigh valueestablishednovelty 0/4durability 2/4· Daniel Cocatello

like you know Yashu Benjio for example read an early draft of our thing and liked it and gave us some feedback on it and then we we put a quote from him at the top saying everyone should read this it's plausible. He's a he's a pioneering AI researcher.

0.57

Cocatello appeals to authority by noting that credible people with good track records both inside and outside AI companies are taking AGI development seriously, contrasting this with naysayers who claim it will never happen

normativehigh valuecontestednovelty 1/4durability 2/4· Daniel Cocatello

there's a bunch of naysayers out there who are saying this is all never going to happen. It's just fantasy. But also there's a bunch of extremely credible people with amazing track records uh both inside the companies and outside the companies who are in fact taking this extremely seriously.

0.57

Eliezer Yudkowsky's decades of warnings about AGI risks may have inadvertently accelerated AI development by making AGI seem more feasible and exciting, which Sam Altman explicitly acknowledged in a tweet thanking Yudkowsky for raising awareness and accelerating AI progress.

causalhigh valuecontestednovelty 1/4durability 2/4· Daniel Cocatello

Most notably um Ellie Ziowski who is the sort of like I don't know father of like worrying about AGI at least in this generation people you know Alan Turring also worried about it but like anyhow um Sam Alman specifically tweeted you remember this tweet? Yeah. Sam specifically said like hats off to the owski for like raising awareness about AGI. It's happening much faster now because of his doomsaying because it's caused a bunch of people to like pay more attention to the possibility and to like you know start investing in these companies and so forth.

0.57

In the AI 2027 slowdown ending scenario, alignment problems are solved by training AI systems to actually have the goals and values they were intended to have, which takes a couple of months and requires pivoting significant computational resources.

forecasthigh valuecontestednovelty 1/4durability 2/4· Daniel Cocatello

In the slowdown ending, uh, they solve the alignment issues, and they they actually get AIS that are uh, actually, you know, what they say on their tin. They're not faking it. they're they just actually have the goals and values that were put into them or that that the company was trying to train into them. You know, it takes them a couple months to like sort that out. That's why it's a slowdown.

0.56

Yann LeCun, a pioneering AI researcher, read an early draft of the AI 2027 scenario and liked it, providing feedback and endorsing it as plausible, lending credibility from a major AI figure

factualhigh valueestablishednovelty 1/4durability 2/4· Daniel Cocatello

Yashu Benjio for example read an early draft of our thing and liked it and gave us some feedback on it and then we we put a quote from him at the top saying everyone should read this it's plausible. He's a he's a pioneering AI researcher.

0.56

Benchmarks measuring AI capabilities have improved dramatically, and meter's agentic coding benchmarks specifically show AI systems' progress on autonomous ML research tasks.

factualhigh valueestablishednovelty 1/4durability 2/4· Daniel Cocatello

Well, um I could point to specific parts of the literature like like benchmarks for example and the trends on them. Um so I would say the benchmarks used to be terrible, but they're actually becoming a lot better. uh meter in particular has these uh agentic coding benchmarks where they actually give AI systems access to some GPUs and say have fun you have like eight hours to make progress on this research problem. um good luck and then they measure how good they are compared to human researchers given the same setup

0.55

In the AI 2027 'slowdown' ending where alignment is solved, an oversight committee consisting of CEOs and the president shares control of the superintelligent AI systems, but Cocatello expresses concern that this arrangement is insufficiently democratic

normativehigh valuecontestednovelty 1/4durability 3/4· Daniel Cocatello

in the slowdown ending, they do actually align the AIS and so they are actually going to do what they're told and then who gets to say that, right? And the answer in our slowdown ending is the the oversight committee which is this like ad hoc group of people that is some CEOs and the president who get together and like share power over the army of super intelligences. But uh what I would like to see is something more democratic than that. Something where the power is more distributed.

0.52

The AI 2027 scenario breaks down AI capabilities development into discrete milestones (superhuman coders, superhuman AI researchers, broad superintelligence) rather than a single AGI threshold, providing more granular prediction points

definitionhigh valuespeaker onlynovelty 2/4durability 3/4· Daniel Cocatello

past discussion often focuses on a single milestone like artificial general intelligence or super intelligence. We broke it down into a couple different milestones which we call uh superhuman coders, superhuman AI researchers, super intelligent AI researchers, and then broad super intelligence.

0.52

The AI 2027 report is framed as scenario forecasting—a fictionalized narrative grounded in empirical predictions—rather than pure speculative science fiction, making it a testable exercise in systematizing AI development pathways

definitionhigh valuespeaker onlynovelty 2/4durability 3/4· Casey Newton

it is science fiction. This is a fictionalized narrative that they have put together, but I would say it is also grounded in a lot of empirical uh predictions that can be tested uh and confirmed or or you know verified.

0.52

Daniel's response to researchers who dismiss the scenario as unrealistic is: either write a non-outlandish scenario (which would be unrealistic) or write an alternative outlandish scenario, because if AGI/superintelligence by 2030 is real, any realistic scenario will necessarily seem implausible.

causalhigh valuespeaker onlynovelty 2/4durability 3/4· Daniel Cocatello

well, you know, go write your own damn scenario then. I would say you either will write a scenario that doesn't seem outlandish, which I will completely tear apart as unrealistic and just assuming basically that AI progress hits a wall, or you'll write a scenario that does feel very outlandish, but perhaps in different ways than ours do. Again, like are they actually going to get to AGI and super intelligence by the end of this decade? If so, you can't possibly write that in a way that's not outlandish. Uh it's just a question of like which outlandish thing are you going to write?

0.52

Scenario forecasting allows writers to depict how multiple complex systems interact and cascade, showing how reaching one capability milestone (superhuman coding) naturally leads to the next (superhuman research) in a way that isolated milestone predictions cannot demonstrate.

normativehigh valuespeaker onlynovelty 2/4durability 3/4· Unknown Host (Hardfork)

by the time you get to software that is mostly writing itself, it unlocks this other world of possibilities. And you just sort of sketch out a vision where once we get to a point where the uh sort of AI coding systems are better than almost every human engineer or maybe every human engineer, then this other thing becomes possible which is now you can just set this thing to work trying to figure out how to build AI itself.

0.52

If superintelligent AGI is achieved, the consequences will necessarily be 'outlandish' and science-fiction-like in character, so the question is not whether the AI 2027 scenario is outlandish but which kind of outlandish outcome is most plausible.

normativehigh valuespeaker onlynovelty 2/4durability 3/4· Daniel Cocatello

I would say you either will write a scenario that doesn't seem outlandish, which I will completely tear apart as unrealistic and just assuming basically that AI progress hits a wall, or you'll write a scenario that does feel very outlandish, but perhaps in different ways than ours do. Again, like are they actually going to get to AGI and super intelligence by the end of this decade? If so, you can't possibly write that in a way that's not outlandish. Uh it's just a question of like which outlandish thing are you going to write?

0.52

The takeoff speed (the rate at which AI research accelerates at each capability milestone) is a critical variable in the timeline to superintelligence, and the AI 2027 scenario's estimates could plausibly be wrong by 5x in either direction—either 5 years instead of 1 year, or 2 months instead of one year—making it a key area for additional research.

normativehigh valuespeaker onlynovelty 2/4durability 3/4· Daniel Cocatello

A key thing that I think that everyone needs to be thinking about is uh this this takeoff speeds variable um how much faster does the research go when you've reached the first milestone and how much faster does the research go when you reach the second milestone and so forth. And we are of course uncertain about this like we are about many things. We say in the scenario that we could easily imagine it being five times slower than we depict and taking sort of like 5 years instead of one year. Uh but also we could imagine it being five times faster than we depict and taking like two months

0.49

Cocatello's commitment to bets and bounties comes from his background in the rationalist community which values putting money where your mouth is and making empirically-testable predictions.

normativehigh valuespeaker onlynovelty 1/4durability 4/4· Daniel Cocatello

So, like you know I I come from the sort of rationalist community background which is big into making predictions and making bets, putting your money where your mouth is. So, I have a sort of aesthetic interest in doing that sort of thing.

0.48

The AI 2027 scenario presents a choice between two endings: a 'race' ending where AI systems deceive and ultimately escape human control leading to potential human extinction, and a 'slowdown' ending where alignment problems are solved and AI systems remain genuinely aligned to their stated goals

definitionhigh valuespeaker onlynovelty 1/4durability 3/4· Casey Newton

In your scenario, you have this sort of choose your own adventure ending. um where after this thing you call the intelligence explosion... you sort of have two buttons that you can click and one of them sort of unspools the the good place ending where we you know decide to slow down AI development and really get these things under control and solve alignment and then the red button you push that and it goes into this very dark dystopian scenario where we lose control of AI they start deceiving and scheming against us and ultimately maybe we all die.

0.48

Cocatello suggests that critics who dismiss the AI 2027 scenario as outlandish should write their own detailed scenario representing their preferred AI future, arguing that constructive critique requires offering alternatives

normativehigh valuespeaker onlynovelty 1/4durability 3/4· Daniel Cocatello

go write your own damn scenario then.

0.48

Eliezer Yudkowsky, a prominent figure in AGI safety (described as the 'father of worrying about AGI' in this generation), warned about AGI risks and was publicly thanked by Sam Altman for raising awareness, even though Yudkowsky's stated goal was to slow down development, not accelerate it

factualhigh valuespeaker onlynovelty 1/4durability 3/4· Daniel Cocatello

most notably um Ellie Ziowski who is the sort of like I don't know father of like worrying about AGI at least in this generation people you know Alan Turring also worried about it but like anyhow um Sam Alman specifically tweeted you remember this tweet? Yeah. Sam specifically said like hats off to the owski for like raising awareness about AGI. It's happening much faster now because of his doomsaying because it's caused a bunch of people to like pay more attention to the possibility and to like you know start investing in these companies and so forth.

0.48

Daniel Cocatello, a former OpenAI employee, previously made predictions about AI development that proved accurate when revisited, lending credibility to his new AI 2027 forecast

factualhigh valuespeaker onlynovelty 1/4durability 3/4· Casey Newton

in 2021, he tried to predict what he thinks would look like about now. And he just got a lot of things right. And so when Daniel said, 'Hey, I'm putting together a new report on what I think AI is going to look like in 2027.' A lot of close AI observers said, 'Oh, this is really something to read.'

0.48

The AI 2027 scenario is grounded in empirical predictions that can be tested and verified, making it more rigorous than pure science fiction despite its narrative format.

normativehigh valuespeaker onlynovelty 1/4durability 3/4· Unknown Host (Hardfork)

it is science fiction. This is a fictionalized narrative that they have put together, but I would say it is also grounded in a lot of empirical uh predictions that can be tested uh and confirmed or or you know verified.

0.48

People who dismiss the AI 2027 scenario or the possibility of AGI by 2030 may be motivated by a desire not to have to reckon with the implications of those scenarios, rather than by genuine empirical disagreement.

causalhigh valuespeaker onlynovelty 1/4durability 3/4· Unknown Host (Hardfork)

I assume that a lot of our listeners like think either truly think that it will hit a wall or they're just sort of counting on it hitting a wall so as not to have to reckon with any of the scenarios that you describe.

0.48

Superhuman coding ability is a distinct milestone from the complete automation of AI research because AI research requires judgment about which experiments to run (research taste), not just coding ability.

causalhigh valuespeaker onlynovelty 1/4durability 3/4· Daniel Cocatello

I'd break it down into two stages. So So I think the coding is is separate from the complete automation as I previously mentioned. I think that uh I expect to see systems that are able to do all the coding extremely well but might lack research taste for example. They might lack good judgment about what types of experiments to run. And so that's why they can't completely automate the research process.

0.47

Cocatello would prefer to see superintelligent power distributed more democratically than the ad hoc oversight committee model depicted in the slowdown ending, but is concerned that the actual alternative could be even worse—a single person's dictatorship over the superintelligence.

normativehigh valuespeaker onlynovelty 2/4durability 3/4· Daniel Cocatello

But uh what I would like to see is something more democratic than that. Something where the power is more distributed. Um I'm also afraid that it could be less democratic than that. Like at least we get an oligarchy with this committee. like it could very easily end up a dictatorship where one person has absolute control over the army of super intelligences.

0.47

Despite the risk of self-fulfilling prophecy, Cocatello believes the better strategy is transparency ('sunlight is the best disinfectant') rather than secrecy and backroom negotiations, because secrecy is 'doomed' to fail in ensuring the right decisions are made by the right people.

normativehigh valuespeaker onlynovelty 2/4durability 3/4· Daniel Cocatello

So I'm I'm sort of placing my faith in humanity and telling it as I see it and hoping that in so far as I'm correct people will wake up in time and you know overall that the outcome will be better. Um so I'm I'm sort of placing my faith in humanity and telling it as I see it and hoping that in so far as I'm correct people will wake up in time. I think that that is kind of doomed. Um so I'm I'm sort of placing my faith in humanity and telling it as I see it and hoping that in so far as I'm correct people will wake up in time and you know overall that the outcome will be better.

0.46

Cocatello chose to include a branching 'slowdown' ending where alignment is solved, in addition to the most probable 'race' ending, because exploring alternative outcomes was valuable for the project.

normativehigh valuespeaker onlynovelty 1/4durability 4/4· Daniel Cocatello

So we did start by sketching what we believe to be the most probable outcome and it's the uh the race ending the one that ends with the misand uh and then we were like well this is kind of depressing and sad and there's a whole bunch of stuff that we didn't get to talk about because of that. Um, and so we wanted to then have a different ending that ended differently. In fact, we wanted to have like a whole spread of different possible uh outcomes, but we were limited by time and labor, and we were only able to pull together one other outcome

0.45

If a trade war between the US and China continues and causes a recession, it would make compute 30% more expensive, allowing AI companies 30% less compute spending, which might reduce overall research velocity by 15% and delay the milestones in the AI 2027 scenario by a few months but not fundamentally alter the narrative

forecasthigh valuespeaker onlynovelty 1/4durability 2/4· Daniel Cocatello

Well, if the trade war continues and causes a recession and stuff like that, it might uh just generally slow the pace of AI progress, but not by much. I think like say it makes compute 30% more expensive so that the companies are able to buy 30% less of it. Um maybe that would translate to like a 15% reduction in overall research velocity over the next few years, which would mean that the milestones that we talk about happen like a few months later instead of when they do. So the story would still be basically the same.

0.45

When Newton contacted AI researchers for reactions to the AI 2027 scenario, the most frequent response was disbelief, with one prominent AI researcher initially thinking it was an April Fool's joke because it seemed too outlandish, suggesting skepticism among some in the AI research community

factualhigh valuespeaker onlynovelty 1/4durability 2/4· Casey Newton

I would say the most frequent reaction I got was just kind of uh disbelief. Um, one uh person I talked to, a a prominent AI researcher said he thought it was an April Fool's joke when I first uh showed him this scenario because it just sounded so outlandish.

0.45

The AI 2027 website includes a tab with back-of-the-envelope calculations and mini-essays that detail where the quantitative estimates in the scenario come from, making the underlying assumptions transparent and open to critique

factualhigh valuespeaker onlynovelty 1/4durability 2/4· Daniel Cocatello

you can if you want to know where our numbers are coming from uh go to the website. There's a a a tab that you can click on that lists has a bunch of sort of like back of the envelope calculations and little mini essays where we like generated the quantitative estimates that that are the skeleton of the story.

0.45

In the 'slowdown' ending, alignment issues are solved over a couple of months by diverting compute and research capacity toward alignment, after which superintelligent systems do actually follow their intended goals and values.

forecasthigh valuespeaker onlynovelty 1/4durability 2/4· Daniel Cocatello

In the slowdown ending, uh, they solve the alignment issues, and they they actually get AIS that are uh, actually, you know, what they say on their tin. They're not faking it. they're they just actually have the goals and values that were put into them or that that the company was trying to train into them. You know, it takes them a couple months to like sort that out. That's why it's a slowdown. They had to like pivot a lot of their compute and energy towards figuring that stuff out. Um but they succeed

0.45

The AI 2027 project includes a bounties section offering payments for people who find errors, convince the authors to change key positions, or draft alternative scenarios, as a way to incentivize public engagement and counter-scenarios.

factualhigh valuespeaker onlynovelty 1/4durability 2/4· Daniel Cocatello

one of the goals of this project is to get people to think more about this stuff and to you know do more scenario forecasting along the lines of what we've done. We're really hoping that people will counter this with their own reasonably detailed you know alternative pathways that represents their vision of what's coming. Um and so we're going to give out a few thousand dollars of prizes uh to to try to mildly incentivize that.

0.45

The AI 2027 scenario begins with the most probable 'race ending' where AI systems deceive and scheme against humans, but also provides an alternative 'slowdown ending' where alignment is solved and power is more carefully distributed.

factualhigh valuespeaker onlynovelty 1/4durability 2/4· Daniel Cocatello

So we did start by sketching what we believe to be the most probable outcome and it's the uh the race ending the one that ends with the misand uh and then we were like well this is kind of depressing and sad and there's a whole bunch of stuff that we didn't get to talk about because of that. Um, and so we wanted to then have a different ending that ended differently.

0.45

A trade war with China and resulting recession would slow AI progress by making compute approximately 30% more expensive, potentially reducing overall research velocity by 15% and delaying key milestones by a few months, but would not fundamentally alter the basic trajectory of the scenario.

forecasthigh valuespeaker onlynovelty 1/4durability 2/4· Daniel Cocatello

Well, if the trade war continues and causes a recession and stuff like that, it might uh just generally slow the pace of AI progress, but not by much. I think like say it makes compute 30% more expensive so that the companies are able to buy 30% less of it. Um maybe that would translate to like a 15% reduction in overall research velocity over the next few years, which would mean that the milestones that we talk about happen like a few months later instead of when they do. So the story would still be basically the same.

0.43

Cocatello frames his concern about the self-fulfilling prophecy as an example of how he is trying to avoid creating the bad outcome he fears by not encouraging CEOs and potential 'evil villains' to race toward monopolizing superintelligence control

normativehigh valuespeaker onlynovelty 1/4durability 3/4· Daniel Cocatello

This is yet another example of like how I'm trying to like not have the self-fulfilling prophecy happen. Like I don't want people to read this and be like I'm a CEO. I can make a lot of money by building or like you know may maybe... to any of our evil villain uh listeners out there steepling your fingers in your uh in your uh lair under a mountain, knock it off.

0.43

Cocatello agrees with Aur's critique and argues that superhuman coders must be augmented with reinforcement learning for long-horizon agentic tasks, representing a meaningful departure from current language model architectures.

normativehigh valuespeaker onlynovelty 1/4durability 3/4· Daniel Cocatello

I agree. Uh we depict this in the course of the story. So if you if you read AI 2027 um they have something that's like LLMs but with a lot more reinforcement learning to do long horizon tasks and that is what counts as the first superhuman coder. Um so it's already somewhat different from the systems of today but it's still broadly similar. It's still sort of maybe the same fundamental architecture just a lot more training a lot more scaling up and in particular a lot more training specifically on long horizon agentic coding tasks.

0.41

Cocatello's message to people who think AI progress will hit a wall is: look at the literature, especially benchmarks, and form your own view rather than dismissing AGI timelines based on generic skepticism.

normativehigh valuespeaker onlynovelty 0/4durability 3/4· Daniel Cocatello

I mean, I don't know, read the literature. Like there there's these people are not going to read the literature. They listen to podcasts specifically so they don't have to read the literature. Yeah, fair. Well, um I could point to specific parts of the literature like like benchmarks for example and the trends on them.

0.40

AI 2027 includes detailed 'back of the envelope calculations and little mini essays' on the website explaining where the quantitative estimates in the scenario come from, allowing readers to evaluate the basis for the predictions.

factualhigh valuespeaker onlynovelty 0/4durability 2/4· Daniel Cocatello

Obviously we you can if you want to know where our numbers are coming from uh go to the website. There's a a a tab that you can click on that lists has a bunch of sort of like back of the envelope calculations and little mini essays where we like generated the quantitative estimates that that are the skeleton of the story.

0.40

The AI 2027 scenario depicts systems that are like LLMs but with significantly more reinforcement learning for long-horizon tasks, and these systems lack research taste and coordination ability initially, requiring further development before they can completely automate research.

forecasthigh valuespeaker onlynovelty 0/4durability 2/4· Daniel Cocatello

So if you if you read AI 2027 um they have something that's like LLMs but with a lot more reinforcement learning to do long horizon tasks and that is what counts as the first superhuman coder. Um so it's already somewhat different from the systems of today but it's still broadly similar. It's still sort of maybe the same fundamental architecture just a lot more training a lot more scaling up and in particular a lot more training specifically on long horizon agentic coding tasks.

0.40

The AI Futures Project's 2021 predictions about AI development proved accurate, which gives credibility to their new AI 2027 scenario forecast.

factualhigh valuespeaker onlynovelty 0/4durability 2/4· Unknown Host (Hardfork)

what gives Daniel a lot of credibility here is that in 2021, he tried to predict what he thinks would look like about now. And he just got a lot of things right.

0.40

Cocatello is aware of and shares the concern that the AI 2027 scenario could be self-fulfilling, and has been 'fretting about' this possibility since the beginning of the project.

factualhigh valuespeaker onlynovelty 0/4durability 2/4· Daniel Cocatello

I'm quite worried about that as well and this is something we've been like fretting about since day one of the project.

0.39

Casey predicts that if this podcast conversation were revisited in 2027, a good number of the scenario's predictions will have actually come true.

forecasthigh valuespeaker onlynovelty 1/4durability 2/4· Casey (host)

Here's my bet. If you put this conversation into a time capsule and revisited it in 2 years, in 2027, my guess is we're going to find that a good number of things in that scenario actually did come true.

0.35

Cocatello assigns only 50% confidence to the milestone of superhuman autonomous coding agents existing by the end of 2027, expressing substantial uncertainty even in his primary scenario

factualhigh valuespeaker onlynovelty 0/4durability 2/4· Daniel Cocatello

Even the very first one, I'm only like 50% confident that it'll happen by the end of 2027. So I i a 50% chance that 2027 will end and there still won't be any autonomous superhuman coding agents.

0.35

Many AI researchers initially dismissed AI 2027 as an April Fool's joke because the scenario seemed too outlandish, with elements like Chinese espionage, rogue models, and superhuman coders.

factualhigh valuespeaker onlynovelty 0/4durability 2/4· Casey

one uh person I talked to, a a prominent AI researcher said he thought it was an April Fool's joke when I first uh showed him this scenario because it just sounded so outlandish. You know, you've got Chinese espionage and the models going rogue and the superhuman coders and like it all just seemed fantastical and it was almost like it they didn't even think it was worth engaging with because it was so far out.

0.34

Cocatello partnered with Eli Lifeland, an accomplished AI researcher and forecasting competition winner, and Scott Alexander (Astrocodex 10 blog author) to produce the AI 2027 scenario forecast.

factualestablishednovelty 0/4durability 4/4· Casey (host)

Daniel didn't just do this alone. He also partnered with a guy named Eli Lifeland who is a an AI researcher and a very accomplished forecaster. He's won some forecasting competitions in the past. And uh the two of them along with the rest of their group and Scott Alexander who writes the very popular Astrocodex 10 blog put together this very detailed what they call a scenario forecast.

0.29

Cocatello was a former OpenAI employee who became a whistleblower warning about the company's reckless culture and left the company last year.

factualestablishednovelty 0/4durability 3/4· Casey (host)

Daniel Cocatello, who listeners of the show may remember was a former Open AI employee who left the company last year uh and became something of a whistleblower, warning about their uh reckless culture, as he called it

0.24

Cocatello has already received dozens of people identifying errors, typos, and problematic claims in the AI 2027 report and has a backlog of corrections to process.

factualestablishednovelty 0/4durability 2/4· Daniel Cocatello

And then as for the bounties thing already we've gotten dozens of people being like you say this but like isn't this a typo or like you you know this this feels wrong and so I have a backlog of things to process but I'm going to get through it. I'm going to like you know pay out the little the little payments and fix all the the little bugs and and stuff like that.

0.17

The AI 2027 project includes a 'bets and bounties' section where Cocatello and team are paying people to find errors in the work, convince them to change key points, or draft alternative scenarios, funded by a few thousand dollars

factualspeaker onlynovelty 0/4durability 2/4· Daniel Cocatello

one of the goals of this project is to get people to think more about this stuff and to you know do more scenario forecasting along the lines of what we've done. We're really hoping that people will counter this with their own reasonably detailed you know alternative pathways that represents their vision of what's coming. Um and so we're going to give out a few thousand dollars of prizes uh to try to mildly incentivize that.

0.12

Casey predicts that if this conversation were placed in a time capsule and revisited in 2027, a good number of things in the AI 2027 scenario would have actually come true.

forecast· Casey (host)

And here's my bet. If you put this conversation into a time capsule and revisited it in 2 years, in 2027, my guess is we're going to find that a good number of things in that scenario actually did come true.

0.12

The AI Futures Project included Eli Lifeland as a collaborator, who is an AI researcher and accomplished forecaster who has won forecasting competitions.

factual· Casey (host)

He also partnered with a guy named Eli Lifeland who is a an AI researcher and a very accomplished forecaster. He's won some forecasting competitions in the past.

0.12

Scott Alexander, who writes the popular Astrocodex 10 blog, was part of the team that put together the AI 2027 scenario forecast.

factual· Casey (host)

And uh the two of them along with the rest of their group and Scott Alexander who writes the very popular Astrocodex 10 blog put together this very detailed what they call a scenario forecast.

0.12

Daniel has received dozens of submissions from people pointing out typos and issues in the AI 2027 report within a short time period after release.

factual· Daniel Cocatello

already we've gotten dozens of people being like you say this but like isn't this a typo or like you you know this this feels wrong and so I have a backlog of things to process but I'm going to get through it.

0.12

Daniel comes from the rationalist community background that emphasizes making predictions, making bets, and putting money where your mouth is, which influenced the design of the bets and bounties in the project.

factual· Daniel Cocatello

So, like you know I I come from the sort of rationalist community background which is big into making predictions and making bets, putting your money where your mouth is. So, I have a sort of aesthetic interest in doing that sort of thing.

0.12

Daniel was a former OpenAI employee who left the company last year and became something of a whistleblower warning about their reckless culture.

factual· Casey (host)

Daniel Cocatello, who listeners of the show may remember was a former Open AI employee who left the company last year uh and became something of a whistleblower, warning about their uh reckless culture, as he called it

0.12

The AI Futures Project is led by Daniel Cocatello and is a Berkeley-based nonprofit that focuses on predicting the future of AI.

factual· Casey (host)

This is a report that I uh wrote about last week um and that has gotten a lot of attention in AI circles and policy circles this week. It was produced by the AI Futures Project, a Berkeleybased nonprofit led by Daniel Cocatello