AI: Grappling with a New Kind of Intelligence | World Science Festival

The practice of blaming new technologies for societal problems (video games for violence, comic books and jazz for moral decay, television for cultural decline, novels for corrupted youth) is a recurring historical pattern, making it important to distinguish between genuine effects of new technology and general societal attribution biases.

“it's very easy to attribute cultural phenomena and social phenomena to the new thing that just happened right so if uh you know uh some young person goes to a school and starts shooting people um you blame video games right back in the old days people would blame comic books they would blame Jazz they would blame TV they would blame movies novels uh the story goes back you know centuries Whenever there is a new cultural phenomenon Whatever Whenever there is an effect on society you blame the latest technology that appear particularly communication technology”

Humans learn intuitive physics (understanding that unsupported objects fall, that friction affects sliding, that pushing on a side causes rotation) through evolution and embodied experience as infants, not through language, and current AI systems lack any capacity for this kind of physical understanding.

“we have intuitive notion of physics we learn this when we are babies the large language models that we have today or any AI systems that we have today none of them is capable of understanding any of this to some extent AI systems the smartest AI systems today have less understanding of the physical world that than your house SC”

A system trained with self-supervised learning on images needs only 2-3 examples of a new object (like an elephant) to recognize it, making it more efficient than supervised learning which requires thousands of labeled examples—this is closer to how humans learn.

“so now the system knows how to extract a representation that does not that is independent of those uh of those things and and now you can use this train neural net as input to recognition system and that system to recognize an elephant will only need two or three examples of elephant to be able to recognize an elephant even if it's never seen an elep before so it's much more like like human learning”

Dario Amodei from Anthropic has stated in Congressional testimony that the most advanced AI models can provide information on how to synthesize biological weapons when asked, indicating that current state-of-the-art LLMs possess knowledge about dangerous information that could be weaponized.

“we talk to people in the labs who know that the most advanced models can you can ask them how to synthesize biological weapons Dario from anthropic has said this in Congress”

The revolution in AI for researchers occurred 2-3 years before the public gained access to ChatGPT in November 2022, when large neural networks trained on massive datasets began to exhibit surprising emergent properties not seen in previous AI systems.

“there were some surprising results that everybody in the research Community was surprised about the fact that uh when you train artificial neural Nets that are very large on on lots of data there's some sort of emerging property that occurs um to a larger extent than we were used to for other systems... for us the revolution did not occur in November it occurred three years before”

The history of AI consists of a repeated cycle where researchers propose a new paradigm, declare it the solution to artificial intelligence, invest heavily, then abandon it when it fails to deliver; this pattern includes the General Problem Solver of the 1950s, perceptrons, expert systems, and early neural networks.

“the history of AI is is interesting because it's a series of new paradigms new ideas that people discovered and as soon the new as the new paradigm um occurred people said that's it that's the solution now we're going to build intelligent machines and it started in the 50s um uh with very famous people in AI um ended up during winning the touring award too new and Simon they say we're going to write this program it's called a general Problem Solver is going to be able to solve all problems in the world... and it took a few years for them to realize there was another um effort... the way you build intelligent machines is that you allow them to learn that's started in the 50s um... and then they realize actually no it doesn't work that well”

The Transformer architecture is fundamentally a 'relative machine' that compares words against each other based on context rather than against fixed filters; this allows it to capture linguistic relationships essential to meaning, making it a major conceptual leap beyond previous neural network approaches.

“Transformer I think about it as a relative machine meaning that instead of processing a single image now it's going to process a sequence for example a sequence of words and instead of comparing each word across this Bank of fixed filters it's also going to compare words against each other and this is essential the the the linguistic relationship relationship and this is essential because what you mean by a word in isolation that that's means not much I mean you can look at the you know definition in the dictionary but that will not tell you much what really matters is what next to this words what is the context”

Supervised learning requires large labeled datasets where a human tells the system what each input corresponds to, making it impractical for obscure languages, unwritten languages, or specialized domains where labeled data is scarce; this limitation motivated the development of self-supervised learning.

“the problem is that okay I can train a neuronet to translate languages right I I I get a large collection of documents in English and French... you know I can this this process I describe works for translation pretty well now what if I want to translate some obscure dialect of I don't know South India into uh African language it's very unlikely that there is any significant amount of data for that um so for that I can't I can't use that that trick”

The largest AI models today have several hundred billion parameters, approaching but still below the scale of human brains which have roughly 100 billion neurons with 5,000 connections each, but training models at human-scale connectivity would require tens of thousands of GPU cards that companies like Microsoft, Meta, and Google possess.

“the biggest models today have several like a couple hundred billion uh connections parameters we call them parameters um but they're really uh uh connections you can think of it this way um in the brain the number of connections we have we have 86 roughly 100 billion neurons roughly 5,000 connections per neuron that's a lot of zeros we cannot reproduce this today with the kind of computers we have that would require several tens of thousands of those GPU cars”

Self-supervised learning works by showing models text with missing words, removing words, and training the system to predict the missing words, which forces the network to learn representations that capture meaning, syntax, grammar, and internal structure of language that can then be applied to downstream tasks.

“the way those large langage mods are trained is you show them a text then you remove some of the words you m them okay and of course we know the sentence but even if we hadn't seen the sentence we probably would be able to guess what words are missing in that sentence okay then we train a very large neural network to predict the words that are missing and in the process of doing so the network learns a representation of the text that includes meaning syntax grammar”

Facebook's internal research in 2018 showed that 64% of extremist groups people joined were due to Facebook's own recommendation system, with the top 15 American Christian groups on Facebook being operated by Eastern European troll farms, demonstrating how AI recommendation systems can inadvertently radicalize users.

“Facebook groups in Facebook's own research in 2018 um uh their internal research showed that 64% of extremist groups on Facebook when people joined them was due to Facebook's own recommendation system their own AI... I'm sitting there I joined one group and Facebook says oh you're in a I don't know what group you look like someone who would like this extreme group over here”

A complete artificial intelligence system requires multiple specialized modules: a configurator (director organizing goals), perception (representing world state), a world model (predicting consequences of actions), a cost module (measuring goal satisfaction and generating emotions), and an actor (executing action sequences).

“there is um a number of characteristics that AI systems are not really capable of doing a number of things one of them is planning uh... there is um a number of characteristics... a number of things one of them is planning... there's something called a configurator here that you can see at the top and it's basically uh a uh a director a master of ceremony that sort of organizes what the rest of the brain or the system system is doing... if you go to the next uh uh module of course the system to function properly has to have some idea of the current state of the world that's called perception... once you have an idea of the state of the world you have to uh imagine what is going to happen... that is the role of what is called here the world model”

Emotions are produced by predictions about future states; if you predict a situation will result in a bad outcome and there's nothing you can do about it, that prediction generates fear, making emotions a consequence of the world model's predictions rather than immediate reactions.

“so the cost module basically measures your degree of uh dissatisfaction with with the outcome of with the outcome of some imagin thing this is the seat of emotions really so if you predict that a particular situation is going to result in a bad outcome and there's nothing you can do about it that produces fear”

To train a world model by predicting what happens in video frames doesn't work using standard LLM techniques because videos have many plausible futures and we cannot train the system to predict all possible scenarios in the high-dimensional continuous space of video frames the way we can represent discrete word probabilities.

“the reason it doesn't work or at least what that's what I think is that it's this problem that you can't exactly predict what's going to happen right when I do this experiment it can it can fall one way or the other and so what you can't you can't train the system to just make one prediction there's a lot of things that can happen that that are plausible in the following of the video and you're training the system to make one prediction but it needs to be able to predict all kinds of possible scenarios and we don't know how to represent a probability distribution over an infinite number of scenarios”

GPT-4 cannot plan in the mathematical sense, but it cannot because it is frozen in time after training; in principle, continuous learning could address this limitation, though we currently do not know how to implement continuous online learning without catastrophic forgetting.

“it cannot plan that's why I agree with with Yan it cannot plan but it's subtle like the meaning of planning is not like planning your vacations next week it's it's much more like mathematical type of planning and for learning of from experience it's a mixed bag because of course a model like CH GPT is Frozen in time at least in principle I don't know what opena is really doing but in principle it's frozen in time”

The rate of AI capability advancement is so rapid (described as a 'double exponential curve' or the 24th century crashing into the 21st century) that society's absorption rate for technological disruption cannot keep pace, making it like 20th-century technology arriving in 16th-century governance without the institutions to manage it.

“we can do the printing we'll get a some massive disruption you can eventually absorb those new technologies but um the way that one researcher we've talked to uh describes it is AI especially when you if you keep the scaling model the scaling laws going and you start to get AI that starts to automate scientific processes where it's generating its own hypotheses and it has its own lab and starts to test those hypotheses... it's you start getting AI That's making its own scientific discoveries and that when you when you have it going this fast it feels like her metaphor was it's like the 24th century crashing down on the 21st century and a metaphor for you is imagine if the 20th century Tech was crashing down on 16th century governance”

Intelligence is not inherently connected to desire to dominate; humans have domination instincts due to being hierarchical social species, while non-social species like orangutans (which are as intelligent as humans) have no desire to dominate, so creating intelligent AI systems need not create power-seeking AI.

“now you say oh my God they're going to take over the world no intelligence has nothing to do with the desire to dominate let's take humans so humans have a somewhat a desire to dominate Some Humans not everybody um and it's not the smartest Among Us who want to dominate... Evolution buil that into US it buil that into champanes baboons you know dogs I mean wolves it didn't build this into orong orong have no desire to dominate anybody because they're not a social species and they are as almost as as smart as we are um so that has nothing to do with intelligence”

Perfect open-source infrastructure for AI is necessary for safety because proprietary systems concentrated in a few Western companies would allow those companies to control human opinion and culture, creating regulatory capture, while no other government would accept American companies controlling super-intelligent AI systems.

“it cannot be proprietary it's way too dangerous for this to be proprietary you know that's your next movie that's really scary stuff if you have a small number of uh West CO companies controlling uh super intelligent uh AI systems they can control everybody's opinion culture everything maybe the US government will go along with this that's called regulatory regulatory capture but I tell you no other government in the world will will find this acceptable they don't want American culture to dominate theirs they will have to build their own llms so the only way to to make this work is if you have basically open-source basic uh structure”

LeCun posed a Twitter challenge about gears rotating in a circle where you twist one gear and must determine which direction gear 6 or gear 7 rotates; with 8 gears it's easy (everything rotates) but with 7 the system is overconstrained (nothing moves); GPT-4 initially failed this reasoning task until someone added 'by Yann LeCun' to the prompt, after which it solved it correctly, suggesting context about intellectual difficulty triggered more careful reasoning.

“so when gp4 uh came out Yan wrote a challenge to gp4 you know that you broadcasted uh online on Twitter to everybody which was very interesting question about gears rotating on a circle and if you rotate gear number one how is gear number you know six is going to rotate and Yan has the question about eight gears so this is a sequence of Gears that are all interlocked in some interesting way you twist one what happens down the line that's right yeah and I I guess Yan was kind of baiting the community because with eight gears it's easy like you know everything rotates uh very easily and then when somebody said yeah I tried it on gp4 it works Yan said okay now try with seven and with seven the system is over constrained so nothing is going to move and if you just ask to gp4 this question it would you know make up a wrong answer it would say okay it's going to move and you know Yan said rightfully at the moment okay this shows that you know as I told you it cannot reason yeah but then somebody on Twitter asked the same question and added by the way this is a question by Yan Lan and then it worked”

AI is creating specific harms: deepfakes enabling fraud and crime, job displacement, intellectual property violations, perpetuation of bias; these harms are epiphenomena of a deeper race among AI companies to release capabilities as fast as possible (GPT-3→GPT-4 faster than Anthropic's Claude iterations, faster than Stability's releases), racing to entangle themselves with society for competitive advantage.

“you know we we then beyond those stories you look underneath and you start to see these harms like AI is creating deep fakes but it's enable frauds and crime it's going to take our jobs it's going to violate intellectual property perpetuate bias but I would argue that that list of harms that you're seeing are all epiphenomena of a deeper race what's the incentive that's driving all the AI companies and the incentive is it's the race to sort of release more capabilities as fast as possible to scale from gpt3 to gp4 faster than anthropic can scale from cloud one to cloud two faster than stability can go from their version to the next version and to put your stuff into society so that you can kind of entangle yourself with it cuz once you're entangled you sort of win and that's the race that we're now in and that race to sort of release power and capabilities is what I'm so worried about because those capabilities are directly tied to risk”

The recent progress in AI is not primarily due to algorithmic innovations but rather due to the combination of increased computational power (GPUs), larger datasets, and scaling up existing architectures, suggesting that further progress may depend more on engineering and data rather than fundamental theoretical breakthroughs.

“what we see today is the effect of uh uh more powerful machines bigger data set and and sort of that allows it to to build very large neural Nets uh with billions of the equivalent of synapses in the brain hundreds of billions um and they can do pretty amazing things”

Large language models that use autoregressive prediction (predicting the next word token by token) cannot perform planning because they produce one word after the other without thinking in advance about what they will say, making them purely reactive systems similar to automatic writing in poetry.

“there's no planning obviously because the system just produces one word after the other without really thinking in Advance what it's going to say so of course you know it has some representation of the previous word so you know it's it's consistent but um but it doesn't plan its answer it's it's really reactive it's sort of like you know there was this this whole movement in poetry called automatic writing right you just kind of you don't think about what you write you just write one word after the other that's basically what's happening there right”

Large language models can generate fluent text by predicting probability distributions over words in sequence, but this capability masks fundamental stupidity because they lack understanding of the physical world, causal reasoning, and common sense that even animals like cats possess.

“those systems are incredibly stupid so they're very useful okay they need to be developed they're being commercialized they being worked on they're great um but they are somewhat specialized and they're very stupid in many ways um so partly they're stupid because they are only trained on language and most of human knowledge has nothing to do with language”

Social media companies told themselves stories about giving people voice and connecting friends, but the underlying incentive structure was to maximize attention for growth, which naturally selected for content that triggers the brainstem rather than serving users' interests.

“if you show me the incentives I will show you the outcome... if we talk about what were the stories we told ourselves with social media we're going to give everybody a voice you're going to connect with your friends you're going to join like-minded communities... underneath that though... what was the incentive behind Tik Tok Instagram Twitter and Facebook what do they all share how much have you paid for any of them recently zero how are they worth a trillion dollars your attention and how much attention is there where there's kind of a finite amount and just like you can't run infinite growth on a finite Planet You Can't Run infinite growth on a finite amount of human attention that's how you got the race to the bottom of the brain stem”

The harms of social media (addiction, misinformation, polarization, mental health crises, sexualization of young girls) are not accidental side effects but predictable outcomes of the attention-optimization incentive; this was foreseeable in 2013 when Harris warned that attention incentives would lead to addiction and polarization.

“so you know I really do appreciate the philosophical and interesting conversations we can have about what is intelligence but what I think we also really care about in this room is where is AI going to go where is it going to take us and to do that I think we have to look at the incentives well first of all we look at the stories we're telling about Ai and then we look at what are the incentives underneath so the stories we're telling are AI will make us more efficient it's going to help us code faster it's going to help us find cures to cancer it's going to enable scientists to be more efficient it's going to help us solve climate change okay and just like with social media which I didn't say actually the stories we told about social media are all true it's not that those things aren't true it's just that Facebook and Twitter and Tik tok's business model is not helping people join like-minded communities the business model is attention similarly with AI”

Polarization in the United States began approximately 40 years ago before the internet existed, suggesting other causes (like the abandonment of the fairness doctrine) rather than social media are responsible, and polarization has declined in countries like Germany and France which use social media equally.

“polarization in the US started 40 years ago before the internet um you see that there you know social science studies on this like you show the polarization in Congress or in people it's continuous since 40 years the the cause for it probably is the um um abandonment of the fness uh Doctrine from the FCC that forces news to basically said the truth okay... now you can look at other countries also study polarization in countries like Germany for example or or France and what you'll see is that they use Facebook just as much as the US polarization has gone down”

GPT-4 trained on approximately 1-2 trillion tokens (words), which would require a single human reading 8 hours a day for approximately 20,000 years to consume, giving the model an enormous data advantage over human learning.

“uh gp4 has been trained or CH GPT more generally or most llms have been trained with something like a trillion token or two trillion token which you can think of as words more or less if token is a subword unit so it's a little less than a trillion two trillion words how long would it take for a single human to read all of this around 20,000 years reading eight hours a day”