How ChatGPT Works Technically For Beginners
What this covers
Corrections: 7:00 MRI scan doesn't measure electricity. EEG does. 26:21 1k -10k times the number of neurons. Don't quote me on this, not too sure.
Source description (no synthesized summary yet).
ChatGPT works by simulating the human brain using artificial neural networks trained on massive amounts of internet text data, combining unsupervised learning to understand context with supervised learning from human judges to generate appropriate responses, but remains fundamentally rigid and energy-intensive compared to biological brains.
- ChatGPT uses two separate neural networks: one for understanding input context (trained unsupervised on internet text) and one for generating responses (trained with human supervision)
- Current AI is rigid because neural network connections are fixed after training and don't change until a new version is released, unlike human brains which continuously adapt
- ChatGPT requires 1.5 years to train and deploy versus 25 years for human brain development, and consumes massive amounts of electricity to run on server farms
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Human brains are the only animals on Earth capable of processing natural language and carrying on conversations, which is why scientists decided to study how the brain works and simulate it in computers rather than trying to manually model conversation with mathematics.
“how did the AI scientists crack this conversational AI so the only it turns out the only animal right that can do this massive you know processing of natural language and having a conversation is humans really we're the only animals that can do this on on planet Earth so scientists decided to uh basically say right so if we can't manually model this with mathematics let's literally just take a look at how our brain works and then simulate that in computer”
Neural network training is the process of starting with random neuron connections and activation patterns, feeding in labeled training data, observing wrong answers, telling the network it's incorrect, and iteratively adjusting neuron activation behaviors until the network begins producing correct answers consistently.
“when you have when you input a image of a bird right they all travel remember some neurons will decide to activate and some neurons will only decide to activate to maybe from here to this one this one and this one and then ignore this one and this one it's it's all random in the beginning so you start with a very random neural network you give it an image of a bird and it gives you the answer of dog that's usually what happens when you start randomly and it will give you the wrong answer every single time right and um the the idea is then you have a training uh data set of like let's say thousand pictures of dog bird and Cat and then every time it gives a wrong answer you basically tell the neural networks that was not the right answer so you need to change your activation Behavior a little bit so they will all change their activation behavior and then you will feed in the images again and then test like or is it or giving the right answer or not and then you just keep on doing that over and over until they begin to more or less start to give the right answer”
Human judges training ChatGPT reject responses that are factually incorrect (e.g., '1+1=3'), responses that provide instructions for harmful actions (e.g., 'how to kill another human being'), and any unethical or immoral content, teaching the response-generating network ethics, morals, and factual accuracy.
“this is like equivalent to school teachers in schools and then the human judges will basically say no that's that's illegal answer like you can't sometimes it's flat out wrong and then the human is like that's a wrong answer uh like I don't know one plus one is not three it's two like the humans are actually telling the chegebt you're wrong and then once it corrects itself and also there's there's other cases where Chacha PT would give an answer to how to kill another human being and then the human judge will be like you don't have more okay so you cannot give this kind of answers reject”
Human babies learn language through unsupervised learning by observing and listening to multiple adults speaking about various topics, finding patterns in this unsupervised linguistic data without any formal grammar instruction, even before formal schooling begins.
“human babies they learn to speak the language even before they go through formal education in school so you can see babies at seven years old 80 years old they they start start talking to their mom and dad but they do it in a very unprofessional way that like uneducated way like very rudimentary but they can still communicate but how are they doing that it's not like Mom and father are sitting there teaching grammar to the kid Mom and Dad just kind of speak to them their uncle and whatever other human beings all speak to that baby and they all talk about different things to the baby sometimes they're talking about the TV show sometimes they're talking about like buying a chocolate from a supermarket whatever whatever the topics may be the human baby is kind of sitting there and just receiving all these information from different other adults and then they start to find patterns”
ChatGPT's context-understanding neural network is trained using unsupervised learning by collecting all text-based content from the internet (blog posts, websites, etc.) without human annotation and dumping it into the neural network, allowing the network to discover linguistic patterns autonomously.
“when they're training that neural networks there's no human involvement they just gather all kinds of information from the internet and then just throw it at the um the neural networks and then the neural networks will start to find rough patterns right so this set of um blog posts I think it's talking about something related to like Tech this I think it's talking about like I don't know traveling I think this is talking about I don't know human like a marriage relationships whatever it start to find these patterns and group the blog posts in all the texts well open air claims that they scraped all the text websites and blog posts and everything that is text based they scraped it everything from the internet and they just dumped it right into this neural network”
After training is complete, ChatGPT's neural networks are fixed and don't change while serving customers, and the AI must wait for a new version release (GPT-4, etc.) to become smarter, unlike human brains which continuously adapt during use.
“once these two parts of the neural networks are trained and and complete right uh it serves the customers which are the users of chatgpt website we just literally go there and just type in chat it serves customers for a period of time until a new version is released so they're working on gpt4 and I don't know how long GPT g54 my guess is that they're gonna follow the same pattern”
A person placed on a deserted island without external connections can use their biological intelligence to solve problems, make decisions, and survive, whereas ChatGPT would cease functioning if disconnected from server infrastructure.
“you can throw somebody into a deserted island and that person will figure his way out uh using his intelligence to survive make decisions face problems solve problems on a remote location without being connected to anything else if that person's brain needs to function just grab a potato and eat it right and you get energy and then you just function”
Natural language like English or Korean is fundamentally more difficult for AI to process than mathematics because natural languages are full of nuances, depend on word sequence grammar, require contextual understanding, and contain homonyms where the same word has different meanings depending on context.
“the reasons why this is difficult is our natural language like English you know I speak Korean um I mean whatever whatever other languages you speak they're not precise language like mathematics um you when you're talking um in natural languages they're they are full of nuances both in the grammatical structure of the language so uh the sequence of the sequence in which the words appear in the sentence matter and then depending on the context of what you're talking about the meaning of the same word that's pronounced the same way uh are different”
ChatGPT's supervised response-generating neural network was trained over approximately six months using human judges who work at normal hours (the speaker assumes nine-to-five), with the training timeline being shorter than unsupervised training because humans get tired and don't work continuously.
“the actual response generating part where the human judges are involved and that takes about six months just working humans I guess I guess the humans work nine to five in that case right they have said because humans get tired right and then that takes about six months”
ChatGPT is completely reactionary—it sits idle on a server waiting for user input, then processes the text the user types, runs it through its neural network program, and generates a response; it has no proactive behavior or ongoing processing.
“this also this program right is a completely reactionary and I say that because once you build this program computer program that's simulating our brain it's kind of sitting there and waiting in a computer server because it and then it waits until it's thrown into a certain conversation so some user logs into chatgpt website and then starts typing like a paragraph or paragraphs of something that they want to talk about with the chat GPT and then at that moment it takes all that text data that user literally typed on their keyboard runs it through the program and then gives out an output”
ChatGPT's architecture consists of two distinct parts: (1) an encoder part that understands the context of the user's input message, and (2) a generator part that produces the response, based on a Google paper that decomposed the conversational task into understanding and generation phases.
“on the left hand side what they did was that's the initial part where when the humans actually type in the input like chat message it would come and hit the initial part of the chat gpt's neural network that tries to understand what's going on and try to understand the context of of the input and then the next part is generating the answer so they've broken it down into understanding what's the input understanding the input and then the next second part is generating the answer”
ChatGPT's neural networks use very complex patterns for understanding input and generating responses, not simple architectures, and the key to ChatGPT's effectiveness is not just the neural network structure but how the networks are trained and optimized.
“the neural networks that are involved in here and here are like based on very complex pattern nothing nothing like this simple is going to be very complex pattern but then it's also the way you um train them”
A case study of a young kitten that had half its brain surgically removed due to injury showed the kitten's brain could completely reorganize itself to take on the functions of the missing hemisphere because the young brain had sufficient neuroplasticity.
“there's even a case where a cat was got into an accident and the surgeon took out half of his brain because it had to be cut out and the rest of it the cat's brain because it was a small kitten like a small baby the kitten kitten's brain was able to reorganize itself completely to take on the responsibility responsibility of the other missing half of the brain with the you know one half of the brain that survives”
Simple feedforward neural networks that pass signals only from left to right through layers perform excellently on image recognition tasks, but completely fail on natural language tasks because language requires fundamentally different connectivity patterns than sequential layer-by-layer processing.
“in this particular design of a neural network where you have initially input layers and then next layer of neurons neck and then activating next layer of neurons activating next layer of neurons and eventually activating output layer of neurons is particularly good for image recognition but it this this particular neural network pattern it's it miserably fails in other tasks especially like natural language”
Scientists, particularly those at Google (which the speaker identifies as more advanced than OpenAI in AI research, with OpenAI learning from Google's papers), discovered that neural networks can have complex connection patterns including feedback loops where outputs from later layers feed back into earlier layers, rather than simple left-to-right signal propagation.
“what the scientists realize um and there's a lot of I think Google is like the most advanced AI scientist it's not even open AI who made chpt actually open AI scientists learn from Google papers so what the Google people realize is that right we can come up with not just this one pattern of neural networks we can kind of structure it so that sometimes you could even have a output of a neural network feeding back to the earlier layer that's that's kind of crazy right”
In contrast, when a mature adult cat experienced a similar brain injury and surgery, the mature brain could not reorganize to compensate for the lost tissue, and the cat died.
“but they took a same case with a mature cat where the mature cat got into an accident and we had they had to carve out a part of the brain to to you know hopefully the cat survives and the mature cat's brain could not reorganize and it died right”
ChatGPT requires massive amounts of electricity to run its servers, with the speaker speculating that AI is consuming 30-40% or more of entire U.S. states' electricity supply in tech-driven regions, representing a significant energy footprint.
“Chacha GPT of course you've seen these days a lot of people are using it and the website is constantly down and we're slow that means it requires massive amounts of electricity to run these servers and it requires a lot of gpus to train the neural networks and all these things eats up a lot of energy at the moment I think AI is consuming a lot of electricity right now I wouldn't be surprised if they're spending like I don't know 30 40 of an entire U.S state electricity consumption into AI in some tech driven States or even more percentage than that it's just massive right”
Understanding how ChatGPT works and what it's capable of and not capable of allows people to better control their emotions about it—managing excitement to avoid disappointment and replacing needless fear with informed perspective.
“you can get a better kind of control of them once you get to know the limitations of what shept is capable of and not capable of because you know even if you're excited you don't want to be over excited and get disappointed to find out the limitations later if you're scared you might you don't need to be scared but you're needlessly getting scared because you don't know what it's capable of”
School education represents supervised learning in humans—unlike unsupervised exposure, school provides explicit guidance through teachers and exam scores, telling the child what is a better answer and what is worse, similar to supervised training of AI systems.
“before going to school they go through all this unsupervised basically fine pattern in chaos phase and then they enter school and then they formally get educated and this is with very strict guidance and the school tells the kid either from the teacher or the exam scores the school tells the kid what is a better answer and what is the worst answer and this is a completely supervised learning right”
A software programmer using ChatGPT for two months has found that 80% of their daily coding work is now generated by ChatGPT and other AI code generation tools, with ChatGPT being the primary tool used.
“I've been using chat GPT for the past two months and I'm a software programmer and now every day eighty percent of my work in terms of coding is all generated code by chatgpt and some other AI code generation tools that I incorporate with chatgpt but mostly it's been chat apt”
Sensory inputs from different body parts (eyes, fingers, nose, touch receptors, smell receptors, optical receptors) are converted from physical stimuli into electrical signals, which are then transmitted to the brain where they trigger patterns of neural activation that vary depending on the nature of the input.
“you give input signals from all parts of your body so your eyes your touches from your fingers uh your smell from your nose everything is converted into um electrical signals from their you know sensing Point like fingers or nose smell receptors whatever and then there's a visual Optical receptors in our eyes whatever it is we our body parts receive that and then it will convert that into electrical signals and then send it to your brain”
Nobel Prize-winning neuroscience research demonstrated that neurons in the brain form non-random patterns of connections depending on which part of the brain they're in: memory regions are structured one way, logical processing regions are structured differently, creating distinct connectivity patterns for different functions.
“it turns out there's different patterns that emerge so depending on the part of the brain that's you know dedicated for memory let's say uh it's structured in a certain way if there's a part of the brain that's dedicated for logical thinking logical processing then it's structured in a different way uh the actual neurons are the same but their connections uh look in some parts of the brain like this and then some other parts of the brain it looks something different”
Human brains are made of organic hardware (biological cells) that enables super energy efficiency because electrical signals are tiny, electricity transfer is very efficient (possibly due to the liquid composition of the brain), and the brain can be fueled by simple food like a potato.
“because it's organic Hardware neurons are just cells right so it's super energy efficient like there's electrical signals flowing my body right now but these are very tiny tiny electrical signals and the transferring of this electricity is very very efficient in our body because probably our body is made up of like liquid and stuff like that I don't know and all you have to do is just I don't know just bite into a potato eat it and then you fuel your body um and then your body just kind of works on low energy”
AI research itself has existed for almost 80 years, but conversational AI proved to be an incredibly difficult task that everyone tried and failed to crack until ChatGPT.
“AI research itself has been around for almost like 80 years so it's a pretty old uh field but the conversational and also during this time everybody tried to crack this conversational Ai and everybody failed”
Image recognition was one of the most successful early AI applications demonstrating the power of neural networks, where scientists trained neural networks to classify images by feeding them thousands of labeled images of dogs, birds, and cats, and the networks learned to activate different output neurons for different image categories.
“one of the most successful AI case that really AI became a thing is image recognition and this is where we started seeing the importance of using neural networks what the scientists did basically is they had a set of images thousands and thousands of images of dogs birds and cats okay and then what they want to see is basically when they present this picture of a bird”
Humans have brains that develop through complex neuroplasticity, where neurons are continuously changing connections, neurotransmitter strengths, and activation behaviors every second throughout life, even after full maturation.
“the neurons are always changing in our human body even after it's fully developed we are constantly having these minor tweaks so it's very fluid our neurons are disconnect and connect and change its activation behaviors and like it always changes every second we live and our neurons are completely autonomous and super energy efficient”
ChatGPT is a conversational AI capable of carrying out intelligent conversations with humans, functioning like the Jarvis character from Iron Man movies where it can understand requests, perform analysis, and engage in back-and-forth dialogue.
“chat GPT is a conversational AI that is capable of carrying out an intelligent conversation with a human so you can think of this as almost like the scene from Iron Man in the Marvel movies where the Iron Man goes hey Jarvis I need to build this”
ChatGPT-4 is expected to use significantly more neurons and connections than current ChatGPT, making it more sophisticated and capable than current versions.
“but chatgpt4 which is the next version is supposed to have way more neurons and connections so it's supposed to be more um sophisticated”
The speaker wanted to understand ChatGPT at a very high level and in layman's terms without deep mathematics, focusing on how it works technically, how it's built, and how it's produced and released, because they couldn't find existing resources that explained it well.
“very high level like I don't want to get into the maths or anything like that I just want somebody to explain to me at a very high level in sort of a like a dummy language layman's terms explain to me how it's built how the scientists are building it and um uh yeah how it's basically produced and I couldn't find anything online that did it really well so I went spent a lot of time learning it and I decided to make a video of my own”