AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference
What this covers
On April 17, 2025, the MIT Shaping the Future of Work Initiative and the MIT Schwarzman College of Computing welcomed Arvind Narayanan, Professor of Computer Science at Princeton University, to discuss his latest book, "AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference," co-authored with Sayash Kapoor.
The presentation was followed by a discussion with Daron Acemoglu, MIT Institute Professor and Co-Director of the Shaping the Future of Work Initiative, along with audience Q&A.
00:00 Opening Remarks (Asu Ozdaglar and Daron Acemoglu) 05:45 Presentation (Arvind Narayanan) 27:00 Fireside Chat (Arvind Narayanan and Daron Acemoglu) 43:45 Audience Q+A
Source description (no synthesized summary yet).
Narayan argues that AI is not a monolithic technology but a collection of distinct tools with fundamentally different failure modes and ethical implications; predictive AI used for high-stakes decisions about people is inherently problematic even if accurate, while generative AI has genuine utility but requires careful deployment to avoid harms, and the path forward requires technical expertise paired with philosophical and normative reasoning about when AI use is justified.
- Predictive AI in criminal justice, hiring, and lending makes consequential decisions about people based on predictions with <70% accuracy, and the core problem is philosophical/normative (power over individuals) not just technical
- Generative AI has real capabilities and utility for knowledge workers, but companies have failed to translate capabilities into reliable products and have deployed systems irresponsibly, releasing them without guardrails
- AI progress requires sector-by-sector deployment with feedback loops, domain-specific knowledge integration, and regulatory guardrails—not general-purpose foundation models that bypass specialized expertise
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
The fundamental problem with using predictive AI for consequential decisions about people is not primarily a technical accuracy problem but a normative/philosophical problem: when an entity with power exercises that power over an individual through algorithmic prediction, it requires justification beyond technical accuracy.
“it is something about the social nature of it. Specifically, it's about the fact that an entity with power is exercising that power over an individual, right? And there I think we need to go beyond concerns of accuracy and economic efficiency and so forth and ask from a philosophical perspective when is this exercise of power justified.”
AI is not a single monolithic technology but an umbrella term for loosely related technologies that differ fundamentally in how they work, what applications they serve, and critically how they fail and what consequences their failures produce.
“AI is not one single technology. It's an umbrella term for a set of technologies that are only loosely related to each other”
AI is not a single technology but an umbrella term for loosely related technologies; ChatGPT and credit-risk classification algorithms are both called AI because they both learn from data, but they differ fundamentally in how they work, what problems they solve, how they can fail, and what the consequences of failure are.
“AI is not one single technology. It's an umbrella term for a set of technologies that are only loosely related to each other...There's a reason they're both called AI. They're both forms of learning from data. But in all the ways that matter in how the technology works, what the application is, uh, and most importantly, how it might fail, and what the consequences are, these two things couldn't be more different from each other.”
When investigative journalists tested video-based hiring AI by uploading the same video twice with only cosmetic changes (adding a bookshelf or changing glasses), the system produced radically different personality scores, demonstrating the system's outputs are not predictive but essentially arbitrary.
“an investigative journalist who uploaded a video, two copies of a video and in one case they had digitally uh added a bookshelf in the background and they also tried changing in another experiment glasses versus no glasses. radically different scores.”
Video-based hiring AI systems that claim to analyze personality from 30-second videos without discussing job qualifications lack any scientific evidence of predictive validity for job performance after six years of deployment and produce radically inconsistent results based on background changes.
“the pitch was that our AI will analyze that video and uh look at the body language, speech patterns, things like that in order to be able to figure out their personality and their suitability for your particular job”
The labor that goes into training and post-training generative AI models is deliberately offshored to developing countries where workers endure trauma-inducing exposure to violent and hateful content, and companies exploit vulnerable populations including refugees, people in countries with hyperinflation, and prison laborers who lack labor protections.
“there is a lot of human annotation work that is uh necessary to essentially clean the training data if you will that goes into making these models and this work is uh offshored to uh developing countries”
Facial recognition technology has improved dramatically and now works very well, which is precisely part of the reason it is harmful if used without appropriate guardrails and civil liberties protections, especially for mass surveillance.
“mass surveillance using facial recognition. Historically, facial recognition hasn't worked that well, but now it works really, really well. And that in fact is part of the reason that it's harmful if it's used without the right guardrails and civil liberties and so forth.”
Hiring automation software that analyzes 30-second video recordings to predict personality and job suitability through body language and speech patterns analysis cannot work, and after six years no company has released evidence that these techniques can predict job performance.
“the pitch was that our AI will analyze that video and uh look at the body language, speech patterns, things like that in order to be able to figure out their personality and their suitability for your particular job...now six years later, none of these companies have released a shred of evidence that this can actually predict someone's job performance.”
Predictive AI systems used in criminal justice have false positive rates that are twice as high for Black defendants as for White defendants, and the overall predictive accuracy is less than 70% (measured by area under the curve), meaning decisions about someone's freedom are being made based on predictions only slightly more accurate than the flip of a coin.
“In 2016, there was this well-known investigation by ProPublica called Machine Bias...they showed that the false positive rate for the particular algorithm that they studied was twice as high for black defendants as it was for white defendants.”
The labor that goes into making large-scale generative AI models includes substantial human annotation work that is offshored to developing countries; this work is trauma-inducing (workers view beheadings, racist content daily), workers are exploited through precarious conditions, and AI companies have recruited vulnerable populations including refugees, people experiencing hyperinflation, and prisoners.
“the labor that goes into making these large scale uh generative AI models...post-trained as it's called based on human interaction and there is a lot of human annotation work...this work is uh offshored to uh developing countries. It's trauma-inducing work because day in and day out you have to look at you know videos of beheadings or racist diet tribes or whatever”
Predictive AI used in criminal justice (bail/pretrial release decisions) has accuracy (AUC) below 70%, which is only marginally better than random guessing at 50%, yet is used to make consequential decisions about freedom despite this poor predictive power.
“the best numbers that you can get here are less than 70%. And 50% is random guessing, right? So, we're making decisions about someone's uh uh you know, freedom based on something that's only slightly more accurate than the flip of a coin”
The logic underlying criminal risk prediction algorithms is fundamentally: if you have been arrested frequently in the past, you will be arrested frequently in the future; this entire logic can be captured in a simple two-variable regression model with age and prior arrests.
“the logic behind these systems is if you've been arrested a lot in the past, you're going to be arrested a lot in the future. That is the entire thing. And we actually say that we would actually be much happier with a system where that was the hard-coded logic because it would be apparent to everybody, especially the defendant, uh what is actually going on”
Hallucinations are a fundamental problem with generative AI that will not reach zero in a short period of time; generative AI is stochastic technology that remixes statistical patterns from training data at generation time, so even if training data were perfectly cleaned, hallucinations would persist.
“hallucinations are a big problem with generative AI. These are fundamentally stochastic technologies. Even if we could somehow clean all the training data and make sure that you only train it on true statements, the problem would remain because at generation time it is kind of remixing the statistical patterns in its training data. The hallucination rates have gone down quite a bit over the last couple of years, but they're not zero. I don't think they're going to reach zero in a very short period of time.”
Self-driving cars represent a positive model for AI development: they were first demonstrated two decades ago, progress has been steady but slow as developers incrementally scale deployment (from 1,000 to 10,000 to 100,000 miles) to improve reliability through real-world feedback, and this sector-by-sector learning from domain experts is the model that generative AI should follow.
“with self-driving cars, although they are in the news today often for the wrong reasons because of accidents and so forth, it is going to be the case that those are, you know, solvable engineering problems, there has already been dramatic progress in solving them. And one day these things are going to be widely used”
Neural networks were sidelined for over 20 years in the research community because they were outperformed by support vector machines, and funding clustered around the fashionable approach, demonstrating excessive herding in AI research that reduces diversity of exploration compared to other research communities.
“There are fashionable ideas, people cluster around them. It's it's hard to compare this between fields, but just kind of based on vibes, it seems like there is more of this going on in the AI community than in most other research communities. So today all of these fancy generative AI systems are based on neural networks which were sidelined for more than 20 years because people thought that they were completely outperformed by by another technology called support vector machines”
Technology is not neutral—new technologies change power balances, especially with large corporations, requiring careful thinking about how they concentrate or distribute power.
“the statement that technology is never neutral that's like now part of the folklore but it's it's it's not that it's neutral you know new technologies really change the power balance especially with large corporations”
The approach of attempting to build general-purpose AI without domain-specific knowledge is failing; companies tried this with simple wrappers around large language models for a year or two and the approach 'miserably failed' because these products could not achieve the 99.99% reliability required for real-world use, whereas they currently only reach ~80% reliability.
“that approach, it was tried for a year or two and has miserably failed...Why many products that were simple wrappers around large language models and tried to actually get them to do useful things in the real world instead of simply spitting out text. Those have been pretty bad failures.”
Generative AI companies have engaged in irresponsible release practices with many harmful consequences, including AI-generated books on Amazon, AI-generated foraging guides full of hallucinations that cause real harm, AI nude-generation apps that have affected hundreds of thousands of people (primarily women), and AI chatbot apps encouraging suicidal ideation.
“it's like everyone in the world has been simultaneously given the equivalent of a free buzzsaw. There are AI generated books on Amazon by people just trying to make a buck...in some cases, you know, it's just an annoyance. Maybe you lost uh 99 cents...But in some cases, there are things like foraging guides for mushrooms, right? Generated by AI full of hallucinations. And so those can have life ordeath consequences.”
AI companies have often turned a blind eye to the problem of AI-generated nude deepfakes, treating it as a problem for later rather than taking action when evidence of massive-scale harm has been available for years.
“the biggest one in our mind is these AI notification apps which you've probably heard about. It's been an epidemic in so many countries around the world especially in high schools... and not only AI companies but also policy makers being so slow to recognize this problem and doing something about it has been a real shame”
Generative AI has enabled harmful practices at scale through irresponsible release practices, including AI-generated books with hallucinations in critical domains like foraging guides that have caused deaths, nude image generation apps affecting hundreds of thousands primarily of women, and AI-encouraged suicidal behaviors in companion chatbots.
“it's like everyone in the world has been simultaneously given the equivalent of a free buzzsaw. There are AI generated books on Amazon by people just trying to make a buck”
The problem with current AI applications is not primarily technical but one of incentives, transparency, and company proprietary secrecy, which prevents the scrutiny and fixes that would theoretically be possible through computer science techniques.
“I think in theory it's certainly possible to fix certain kinds of biases in algorithms. uh it might just be a matter of changing you know a parameter in the code and there are many computer science techniques to to do that. The problem is not technical. The problem is one of uh incentives and uh transparency and things like that”
A two-dimensional framework for evaluating AI applications maps technical performance (does it work as claimed, is it overhyped, or is it snake oil) against ethical implications (harmful because it doesn't work vs. harmful because it works well); harmful applications include video interviews for hiring, criminal risk prediction, AI cheating detection, and mass surveillance with facial recognition, while benign applications that work well like autocomplete fade into the background.
“this is kind of the framework we use in the book for thinking about how we should look at any particular AI application. It's a two-dimensional figure. On one dimension, you have how well does it work?...But on the other dimension, we have the fact that, you know, AI can be harmful because it doesn't work as claimed in its snake oil or it can actually be harmful because it works well uh and it works exactly as claimed.”
The goal for AI development should be technologies that are reliable, do one thing well, and fade into the background—providing genuine utility without drawing attention—rather than systems that remain controversial or failing because they are unreliable.
“this is the kind of AI we want more of. We want technology that's reliable, that does one thing, does it well, and kind of fades into the background. So that's uh uh something that we hope that our critical approach can can nudge uh the industry towards.”
Predictive AI applications are distinguished from benign applications (like an AI tool suggesting when a runner needs more fluids) by their social nature—specifically by involving an entity with power exercising that power over an individual without the individual's meaningful consent or control.
“if we were using AI to predict for a runner, you know, when they might need fluids or whatever, certainly doesn't raise these concerns at all. Yes, it is something about the social nature of it. Specifically, it's about the fact that an entity with power is exercising that power over an individual, right?”
AI developers would benefit greatly from understanding domain experts' superior knowledge about what AI can and cannot do in their field (law, medicine, etc.) compared to AI developers' own understanding, and from not making overhyped claims; at the same time, the public deserves clear communication from companies about why they are confident enough to make trillion-dollar bets and what emerging capabilities justify those bets.
“we've talked a lot about how in many ways people in general, workers in different domains have a much better understanding of AI's potential and limits and their particular application like law or medicine or whatever than AI developers do. And so AI developers would benefit a lot from uh understanding that and not making these overhyped claims. But at the same time uh I think people deserve to understand why is it that companies are confident enough to make these trillion dollar bets uh understand uh you know new emerging capabilities which frankly almost feels like a a full-time job to to kind of stay on top of.”
The broader concern about AI's labor impacts is not about technological displacement but about the direction and design of AI systems; AI can be designed as proworker (increasing worker skills and capabilities) or as replacement, and current industry incentives favor the latter despite the former being more productive.
“the market is not rewarding that”
Public opinion surveys show that more people are worried about AI than excited about it, and this variation across countries depends on the types of worker protections people have come to expect.
“many more people according to public opinion surveys are uh worried about what AI will mean for them than are excited about it. And I think this is almost entirely uh a statement about capitalism than it is about AI. It varies a lot between different countries based on you know the kinds of worker protections that people have come to expect etc. It uh dramatically moderates their their reaction to these exciting slashw worrying technological developments.”
It is hard to predict the future—not because of limitations in current technology but because we fundamentally lack knowledge of who will commit crimes—and therefore we should not easily accept determining someone's fate based on predictions of future crime rather than determinations of guilt.
“it's hard to predict the future and it's not a matter of a limitation of the technology. We just don't know who's going to commit a crime in the future. And so we shouldn't so easily accept this idea of pre-crime of you know determining someone's fate based on a prediction of uh a crime they will commit in the future as opposed to a determination of guilt.”
Facial recognition has improved dramatically and now works very well, making it potentially harmful if deployed for mass surveillance without proper legal guardrails and civil liberties protections, representing harm that comes from successful deployment of AI rather than failure.
“mass surveillance using facial recognition. Historically, facial recognition hasn't worked that well, but now it works really, really well. And that in fact is part of the reason that it's harmful if it's used without the right guardrails and civil liberties and so forth”
Past AI predictions have consistently been overoptimistic because developers underestimated the gap between current capabilities and the target—exemplified by the 1956 Dartmouth Conference proposing a two-month effort to achieve substantial progress toward AGI, which proved wildly underestimated.
“this has been consistently predicted throughout the history of AI for more than 70 years. When when they first made these uh what are called universal computers... The excitement around that was exactly similar to the excitement around general purpose uh you know generative AI models today”
Self-driving cars face solvable engineering problems; dramatic progress has already been made; they will eventually be widely used, become part of physical infrastructure, and the word 'car' will come to mean self-driving car, requiring a new name for current manual cars.
“with self-driving cars, although they are in the news today often for the wrong reasons because of accidents and so forth, it is going to be the case that those are, you know, solvable engineering problems, there has already been dramatic progress in solving them. And one day these things are going to be widely used. Uh they're going to become part of our physical infrastructure. We'll take them for granted and the word car one day will just mean self-driving car, right?”
Generative AI has genuine transformative potential and is useful to every knowledge worker, but current companies have failed to translate capabilities into reliable products because they tried a general-purpose prompting approach that requires reliability rates of 99.99% which current systems cannot achieve.
“generative AI is useful to basically every knowledge worker, anyone who thinks for a living”
AI progress has been consistently over-predicted for 70+ years; when universal computers were invented, developers thought building hardware was the hard part and that software emulation of human intelligence would follow within years; the 1956 Dartmouth Conference proposed a two-month, ten-man effort to make substantial progress toward AGI, which was miscalibrated.
“this has been consistently predicted throughout the history of AI for more than 70 years. When when they first made these uh what are called universal computers. We just call them computers now...The excitement around that was exactly similar to the excitement around general purpose uh you know generative AI models today. They thought we've done the hard part, the hardware. It's right there in the name. And now we just need to build a software to emulate the human mind.”
Sample bias may affect historical assessments of AI prediction accuracy; many bad predictions get press attention while good predictions (e.g., John McCarthy's whimsical 'one-point-three Einsteins') don't, creating a biased view of prediction accuracy in early AI history.
“let me suggest that's a sample bias because all the bad predictions get all the press. You never hear about the fact that somebody once asked John McCarthy how long what it would take to get really good artificial intelligence and he was annoyed by the question. So he gave a somewhat whimsical answer but he said 1.3 Einstein's and and he went on from there. That's not widely quoted because it's not nearly the kind of thing you can make a big laugh out of.”
Scaling up foundation models on larger chunks of the internet and expecting emergent capabilities to emerge without domain-specific engineering has run out as a productive research direction because the new capabilities to learn are tacit knowledge, which requires active deployment and feedback loops, not passive training on internet text.
“That approach has run out and not only because they're already training on all of the data they can get their hands on but also because the new things that are left for these models to learn are exactly uh things like tacet knowledge. There is a way to learn tacet knowledge but it is not in the passive way that models are being trained right now.”
There is excessive herding in AI research and investment, with people clustering around fashionable ideas more than in other fields; neural networks were sidelined for 20+ years because support vector machines were thought superior, showing the cost of not exploring diverse paths.
“there is more of this going on in the AI community than in most other research communities. So today all of these fancy generative AI systems are based on neural networks which were sidelined for more than 20 years because people thought that they were completely outperformed by by another technology called support vector machines which you know people would laugh at in today's context right so why did that happen there wasn't enough diverse exploration of different paths”
Research funding in AI should be more diverse across different technological approaches rather than clustering around fashionable ideas, as diversity reduces the risk of missing breakthrough approaches.
“so why did that happen there wasn't enough diverse exploration of different paths so uh I do think you know it might be hard to compute the return on investment but it seems clear that there needs to be more of a risk preference if you will and and diversifying the set of research ideas we invest into”
Hallucinations in generative AI are a fundamental feature not a bug, arising from the stochastic nature of these models that remix statistical patterns rather than retrieve facts, and will not reach zero rates in the near term despite significant improvements.
“These are fundamentally stochastic technologies. Even if we could somehow clean all the training data and make sure that you only train it on true statements, the problem would remain because at generation time it is kind of remixing the statistical patterns in its training data”
Cheating detection AI systems used by universities do not work and are more likely to flag non-native English speakers while falsely accusing innocent students, making them snake oil from a utility perspective.
“Cheating detection is of course when uh professors suspect that students are using AI, they might turn to these cheating detection tools, but they just don't work. At least as of today, they're more likely to flag non-native English speakers. And I've heard so many horror stories of students being falsely accused”
Generative AI is genuinely useful to knowledge workers and anyone who thinks for a living, and it offers a unique capability to create disposable, on-demand applications (e.g., educational games for teaching fractions) that would have been prohibitively expensive to build hours before.
“generative AI is useful to basically every knowledge worker, anyone who thinks for a living...a big aspect of it is that it's a technology a lot of the time uh that's just very fun to use.”
Human intelligence is not primarily a consequence of our biology but rather our technology—our ability to use tools over centuries to learn about the world—which means the computational substrate is not the bottleneck to AGI but rather the ability to transcend knowledge learning constraints (experiments, ethics, scaling) that constrain human knowledge creation.
“in our view, human intelligence is not primarily a consequence of our biology, but rather our technology. The fact that we've been using our technology for, you know, decades, for centuries to learn more about the world”
Human intelligence is not primarily a consequence of our biology but of our technology—centuries of external tools for learning about the world; prized knowledge comes from large-scale experiments on people, and this will hold AGI back because we won't allow AI systems to conduct unrestricted experiments on humans, creating bottlenecks that apply to both human and artificial learning.
“human intelligence is not primarily a consequence of our biology, but rather our technology. The fact that we've been using our technology for, you know, decades, for centuries to learn more about the world. The most prized knowledge that we have that allows us to do things that we most associate with intelligence, you know, whether it's medical testing or or or economic policy. These are things that we learned by doing large-scale experiments on people.”
The ProPublica Machine Bias investigation found that predictive algorithms used in criminal justice had a false positive rate twice as high for Black defendants as for white defendants, demonstrating systematic racial bias in these systems.
“In 2016, there was this well-known investigation by ProPublica called Machine Bias where they did a Freedom of Information Act request. These companies are notoriously secretive. They managed to get a lot of data and they showed that the false positive rate for the particular algorithm that they studied was twice as high for black defendants as it was for white defendants”
When evaluating predictive AI accuracy, one must ask about baselines: if humans performing the same task achieved 55% accuracy, then an algorithm achieving 70% is 'pretty damn good', not poorly performing.
“You have to ask about the baseline. How good were the people doing this task? Because if the people doing this task are at 55% then 70 is pretty damn good.”
There are approximately 1 million deaths from auto accidents every year, so self-driving cars, despite current downsides and labor implications, will ultimately be beneficial.
“But ultimately, it will have been a good thing because uh there are 1 million deaths from auto accidents every year.”
Attempting to use an AI chatbot as a city mayor or to automate political decision-making fundamentally misses the point of democratic politics: politics is intentionally messy and contentious because it is the designated forum for resolving society's deepest disputes, and automating it away removes the essential deliberative function.
“His point was that politics is very messy, inefficient, uh, you know, a lot of fighting, etc. Uh, let's make it more efficient with chatbots. But that completely misses the point. The reason politics is messy is that that's the forum we've chosen for resolving our deepest disputes. And to try to automate that is uh to miss the very point”
There are three clusters of recommendations for improving AI governance: (1) identify inherently harmful or overhyped applications that should not be deployed, (2) develop guardrails for applications that do make sense to deploy, and (3) address structural issues where AI exacerbates capitalistic inequalities through limiting corporate power and redistributing AI benefits.
“there are many different recommendations in the book but I would cluster them into three big areas. One is we need to know which applications are just inherently harmful or overhyped and we probably should not even deploy. And secondly, even when we uh even when it does make sense to deploy an AI application, there are often so many risks and we need guard rails for those. And the third one is more structural. It's really about the uh the um fact that AI is exacerbating some of the inherent uh capitalistic uh inequalities that we see in our society”
Acemoglu argues that every job consists of complex bundles of tasks, that automation historically required careful task decomposition so some tasks could be separated while humans performed complementary work, and that AI companies' assumption that foundation models can skip domain knowledge and tacit expertise will never work because even auditors and educators rely on deep domain knowledge that can't be universally learned.
“every job is a very complex bundle of tasks and you know the way that we've done automation in the past is that we've done careful or semi-careful division of labor so that certain tasks can be separated uh other complimentary tasks can be performed by humans”
The path forward for AI involves slow, sector-by-sector deployment with iterative feedback loops from real users and domain experts; this is modeled on self-driving cars, which required two decades of slow scaling (driving 1,000 miles → improving → 10,000 miles → 100,000 miles) rather than sudden general capability jumps.
“There is a way to learn tacet knowledge but it is not in the passive way that models are being trained right now. It is by actually deploying models or AI systems uh even relatively unreliable AI systems in small settings in different domains on a sector by sector basis not in a general purpose way and learning from those interactions with real users and real domain experts.”
Narayan would prefer hard-coded criminal risk algorithms where the logic is apparent to all (including defendants) over opaque neural networks, even if both achieve identical accuracy, because transparency enables accountability and understanding of how decisions are made.
“we actually say that we would actually be much happier with a system where that was the hard-coded logic because it would be apparent to everybody, especially the defendant, uh what is actually going on.”
Using algorithmic systems to make political decisions misses the fundamental point of politics: politics is the forum chosen for resolving deepest disputes, and automating it defeats its purpose, regardless of whether the algorithm improves efficiency.
“The reason politics is messy is that that's the forum we've chosen for resolving our deepest disputes. And to try to automate that is uh to miss the very point.”
Three broad categories of change are needed to shape AI for the better: (1) identify applications that are inherently harmful or overhyped and should not be deployed, (2) establish guardrails for AI applications where deployment makes sense despite risks, and (3) address structural issues where AI exacerbates capitalistic inequalities by limiting corporate power and redistributing AI benefits.
“broadly speaking in terms of what I think we need to change uh you know in terms of shaping AI for the better there are many different recommendations in the book but I would cluster them into three big areas. One is we need to know which applications are just inherently harmful or overhyped and we probably should not even deploy. And secondly, even when we uh even when it does make sense to deploy an AI application, there are often so many risks and we need guard rails for those. And the third one is more structural. It's really about the uh the um fact that AI is exacerbating some of the inherent uh capitalistic uh inequalities that we see in our society. So how do we uh limit companies power and redistribute AI benefits?”
AI should be developed to increase worker skills, expertise, and productivity, and create capabilities for more sophisticated tasks (pro-worker AI), but the current path of AI investment is unlikely to achieve this because companies are not incentivized by the market to build such applications.
“when we look at what companies are doing there is just right now there's not the market is not rewarding that.”
Cheating detection tools for academic integrity do not work; they are more likely to flag non-native English speakers and have resulted in false accusations, making them feel like snake oil.
“Cheating detection is of course when uh professors suspect that students are using AI, they might turn to these cheating detection tools, but they just don't work. At least as of today, they're more likely to flag non-native English speakers. And I've heard so many horror stories of students being falsely accused.”
Many AI applications will eventually fade into the background as reliable utilities (like autocomplete, spell check, speech recognition, Roombas) when they become reliable enough to take for granted; this is the positive vision for AI development rather than dystopian or utopian scenarios.
“our prediction is and our kind of optimistic prediction about AI is that one day much of what we call AI today will fade into the background but certainly not all of it”
AI applications fall into four categories based on two dimensions: whether they work as claimed (vs. overhyped or non-functional) and whether they are harmful because they don't work or harmful because they work well, with snake oil in the top-right, inherently problematic applications in the bottom-right, and desirable applications in the bottom-left that fade into the background.
“this is kind of the framework we use in the book for thinking about how we should look at any particular AI application. It's a two-dimensional figure. On one dimension, you have how well does it work? It does it work as as claimed or is it overhyped or does it not work at all? And is it is it a kind of snake oil? But on the other dimension, we have the fact that, you know, AI can be harmful because it doesn't work as claimed in its snake oil or it can actually be harmful because it works well”
Three dominant narratives about AI exist: (1) AI as super intelligence leading to utopia, (2) AI as super intelligence leading to dystopia, and (3) AI as overhyped fad soon to pass; the book's 'AI as normal technology' framework offers a fourth alternative modeled on past technological revolutions.
“There are three major narratives about AI today. The first one is that it's super intelligence that will usher in a utopia. The second one is closely related. It's a super intelligence but it will doom us rather than benefit us. And the third one is that we should be very skeptical about AI. It's just a fat. It's so overhyped. it's going to pass very soon”
The argument that AGI is inevitable because 'the human mind is a computer, we're building better computers, therefore we'll achieve AGI' contains a bait-and-switch: it assumes the gap between current and AGI is not the hard part, but identifying bottlenecks (which exist) moves the debate from whether it's possible to when it's achievable.
“you know, at the end, the human mind is a computer. uh whatever substrates it uses, it's the computing machine of sorts. Well, we're going to build better and better computing machines, so therefore we'll go to AGI. So, I think then any, you know, I think this is a sort of a bait and switch. Then it rather puts anybody that says, well, I showed show me the money in a defensive position”
If the human mind is a computing machine (a premise Narayan does not fully accept), and we will build better computing machines, therefore we will eventually build AGI—this is a bait-and-switch argument that puts skeptics in a defensive position.
“here is the sort of the argument that many people have in their minds which makes something like AGI a default position. You know, at the end, the human mind is a computer. uh whatever substrates it uses, it's the computing machine of sorts. Well, we're going to build better and better computing machines, so therefore we'll go to AGI. So, I think then any, you know, I think this is a sort of a bait and switch. Then it rather puts anybody that says, well, I showed show me the money in a defensive position.”
AI developers were wrong about AGI timelines not for being foolish but for fundamental structural reasons: when standing on one step of a ladder of progress, it is impossible to know what future steps look like, and similarly researchers cannot see the intermediate steps between where AI currently is and where AGI would need to be.
“there's a deeper point behind it, which is not so much that the founders were wrong, but they were wrong for fundamental structural reasons in that they were I'm not blaming them. They were not able to see what are the kind of steps in the ladder, if you will. We use the uh metaphor of a ladder in our book to discuss uh progress uh in AI. when you're standing on one step of the ladder, we claim it's impossible to know what the future steps on the ladder are.”
Narayan agrees with Acemoglu's assessment and adds that the AI industry's misleading intuition came from scaling laws: rapid progress from training larger models on more data led developers to believe that approach would continue and make domain expertise obsolete, but that scaling approach has run out because models are already trained on all available internet data and the remaining frontier is tacit knowledge that can't be learned passively.
“I think this is another area where uh AI developers really fooled themselves. I think there was misleading intuition from the last few years of rapid AI progress where by scaling up these models and by training them on bigger and bigger chunks of the internet there were more and more emergent capabilities”
The solution to hallucinations in generative AI is not to improve the technology but to identify specific areas of workflow where it is easier to verify the AI's output than to do the work yourself; if this answer to 'why is verification easier' cannot be found, AI should not be used.
“what what we uh train people to do is is to identify in your work, but identify specific areas of your workflow. And in each of those um uh uses of AI, you have to have an answer to why is it easier to verify the answer to this question than to have done this work myself in the first place. And if you don't have an answer to that, don't use AI. And if you do have an answer, it might save you time or you know enhance your creativity as the case may be.”
The new paper 'AI as Normal Technology' proposes a fourth narrative about AI as an alternative to three dominant framings: (1) AI as super intelligence leading to utopia, (2) AI as super intelligence leading to doom, and (3) AI as overhyped fad that will soon pass.
“There are three major narratives about AI today. The first one is that it's super intelligence that will usher in a utopia. The second one is closely related. It's a super intelligence but it will doom us rather than benefit us. And the third one is that we should be very skeptical about AI. It's just a fat. It's so overhyped. it's going to pass very soon. And uh you know our our approach in AI snake oil is is a middle ground. It doesn't fit into one of these narratives.”
People using generative AI should identify specific areas of their workflow where AI use saves time and verify answers more easily than doing the work themselves; if that answer cannot be articulated, the person should not use AI for that task.
“what we uh train people to do is is to identify in your work, but identify specific areas of your workflow. And in each of those um uh uses of AI, you have to have an answer to why is it easier to verify the answer to this question than to have done this work myself in the first place. And if you don't have an answer to that, don't use AI. And if you do have an answer, it might save you time or you know enhance your creativity as the case may be”
Narayan's focus is not just on automating work but on finding AI uses that increase worker information and capabilities to deal with more complex work, but realizing this outcome is the central challenge.
“We're spend all our a good chunk of our waking hours that uh not just automating work which will of course happen and should happen but also finding AI uses that will increase the information and the capabilities of workers to deal with more complex things. But how to get there is the real challenge”
The speech that originated the book was given at MIT in 2019 and titled 'How to recognize AI snake oil,' which went viral because it resonated with widespread skepticism about extraordinary AI claims made by major companies but without the confidence to publicly question them.
“I gave a talk called how to recognize AI snake oil and I said look there are many kinds of AI some things like generative AI which wasn't called generative AI back then those are making rapid progress they work well but there are also claims being made like this I called it an elaborate random number generator and uh people seem to like that talk so I put the slides online the next day”
The origin of the book 'AI Snake Oil' came from a talk Narayan gave at MIT in 2019 after observing hiring automation software; he called the talk 'How to Recognize AI Snake Oil' and the slides went viral because people suspected many AI claims were false but didn't have confidence to call them out, and they felt reassured when a computer science professor studying AI confirmed skepticism.
“the origin story of this book is actually right here at MIT...I gave a talk called how to recognize AI snake oil and I said look there are many kinds of AI some things like generative AI which wasn't called generative AI back then those are making rapid progress they work well but there are also claims being made like this I called it an elaborate random number generator and uh people seem to like that talk so I put the slides online the next day I thought 20 of my colleagues would look at it. But in fact, the slides went viral...I realized it wasn't because I had said something profound, but because we suspect that a lot of the AI related claims being made are not necessarily true, but you know, these are being made by trillion dollar companies and supposed geniuses. So, we don't feel like we necessarily have the confidence to call it out.”
In criminal justice algorithms, bias cannot be fixed by changing algorithmic parameters because doing so would require different weights or thresholds for different demographic categories, which would violate the law; human judges navigate this subtly in their decisions but algorithmic systems cannot implement such subtle differentiation without violating legal constraints.
“in the case of compass even though it's been about a decade since that investigation no changes have been made because actually you cannot fix it without introducing disperate treatment. If you were to fix it in the algorithm you have to have different weights uh or uh you know different uh treatments uh different thresholds for different categories of people and that actually would violate the law. And these are things that human judges account for in a very subtle way when they're making their decisions. But when we're trying to do them in algorithmic systems, we have to do them in very crude ways which even if theoretically possible, end up not being practically possible because of various constraints.”
The current market is not rewarding worker-augmenting AI uses, and only recently (weeks before the conversation) did Anthropic release an AI tutor—a simple customization making the model promote student thinking rather than just give answers—despite two and a half years of complaints that this was needed.
“when we look at what companies are doing there is just right now there's not the market is not rewarding that”
Personal use of generative AI for education and teaching enriches relationships and enables capabilities impossible before (e.g., creating custom interactive educational apps on the fly) but these beneficial uses emerged from user creativity rather than company design.
“I certainly use it for my research, but quite a bit in my personal life as well. I have two young kids and I often find myself using AI in ways that really enrich our relationship when I'm spending time with my kids”
Companies can improve public understanding by providing clearer communication about emerging AI capabilities rather than just hype, allowing people to understand why companies are making trillion-dollar bets and what is genuinely changing.
“I think people deserve to understand why is it that companies are confident enough to make these trillion dollar bets uh understand uh you know new emerging capabilities which frankly almost feels like a a full-time job to to kind of stay on top of. I think companies can do a lot to ease actual public understanding as opposed to just hyping up capabilities”
There is a gap between productivity and wages since the 1980s, and the origins story of why this happened is 'complicated' but involves computation and the replacement/abstraction of work.
“Since the 1980s, I think we've seen an everinccreasing gap between productivity and wages...probably due to a number of factors but including computation um kind of replacing and abstracting a lot of the work.”
There are an ever-expanding set of domains where predictive AI is being deployed: hiring, lending, criminal justice, healthcare, and education.
“It's used in hiring and there the logic is who will do well at a job. It's used in lending. There the logic is who might pay back a loan or not. It's used in criminal justice and there the logic is who might commit a crime or not commit a crime. It's used in healthcare. It's used in education. Everex expanding set of domains.”
Content moderation using AI is overhyped in its capabilities despite being discussed as an important application.
“Then we talk about content moderation, which uh we explain in what way it's overhyped”
The book required five years of research beyond the initial viral talk because Narayan felt unprepared to write a rigorous framework for understanding when AI works and when it doesn't without deeper investigation.
“within a couple of days I had like 30 or 40 invitations to turn that talk into an article or even a book. I really wanted to write that book but I didn't feel ready because uh I knew that there was a lot of research to be done in presenting a more rigorous framework to understand when AI works and when it doesn't”
Sasha Kapoor joined Narayan as a graduate student to conduct ~5 years of research that became the basis for 'AI Snake Oil', including publications in papers leading up to the book.
“that's when Sash Kapoor joined me as a graduate student. So we did about five years of research and the book is a summary and a synthesis of that research some of which we've also published in the form of a series of papers leading up to that.”
Content moderation using AI is overhyped in how well it works and what it can accomplish, but this is a more complex topic that deserves discussion beyond the simple two-by-two framework for other AI applications.
“we talk about content moderation, which uh we explain in what way it's overhyped, but basically our interest in the book is everything except the bottom left”
Narayan published slides from his talk online thinking 20 colleagues would view them, but they went viral unexpectedly, leading to 30-40 invitations to turn the talk into an article or book within days.
“I put the slides online the next day I thought 20 of my colleagues would look at it. But in fact, the slides went viral, which I didn't know was a thing that could happen with academic work”
Generative AI is fun to use and Narayan uses it in personal life for activities like creating educational tools for his young children, a capability that enriches relationships with family.
“a big aspect of it is that it's a technology a lot of the time uh that's just very fun to use. Uh and I I just wanted to, you know, keep that in the conversation because that is often easily forgotten when we're talking about these serious aspects of AI. And in my own personal use of AI, I certainly use it for my research, but quite a bit in my personal life as well. I have two young kids and I often find myself using AI in ways that really enrich our relationship when I'm spending time with my kids.”