What this covers
Veritasium traces how a 19th-century Russian mathematical disagreement spawned tools that now power everything from nuclear simulations to Google search and text prediction. Andrey Markov and Nikolai Nekrasov clashed over whether the law of large numbers—the statistical principle that averages stabilize over time—requires independent events. Markov settled the matter by analyzing 20,000 letters from Pushkin's Eugene Onegin, showing that even though vowel-consonant pairs were heavily dependent on each other, they still converged to predictable ratios. His insight led to Markov chains, a mathematical framework modeling systems where the next state depends only on the current one, not the full history before it.
The video follows how this one-century-old proof became foundational to three modern domains. In nuclear physics, John von Neumann realized neutron behavior couldn't be simulated independently and built the Monte Carlo method—running random simulations through a Markov chain to approximate solutions too complex for equations. In web search, Larry Page and Sergey Brin modeled the internet as a Markov chain where surfing patterns reveal page importance, yielding PageRank and upending early search engines that ranked by keyword repetition alone. In language, the video shows how predicting the next letter or word in a sequence is itself a Markov chain prediction problem, and how modern large language models extend this with attention mechanisms to handle longer context. The presenters also cover surprising applications: why shuffling a deck seven times suffices for randomness, and why feedback loops—like AI training on AI-generated text—become hard to predict with Markov chains.
A century-old feud between mathematicians Markov and Nekrasov over whether the law of large numbers requires independent events produced Markov chains, a memoryless probabilistic tool that now underpins the Monte Carlo method, Google's PageRank, and large language models.
- Markov disproved Nekrasov's claim that convergence in statistics implies independent (free-will) decisions by showing dependent events (letters in a poem) also obey the law of large numbers
- The memoryless property of Markov chains lets complex history-laden systems be modeled using only their current state
- Markov chains became foundational to nuclear simulation (Monte Carlo), web ranking (PageRank), and text prediction (LLMs)
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Markov analyzed the first 20,000 letters of Pushkin's 'Eugene Onegin' and found vowel-vowel pairs occurred only 6% of the time, far below the 18% expected if letters were independent, proving the letters were dependent on one another.
“if the letters were independent, the probability of a vowel-vowel pair would just be the probability of a vowel twice, which is about 0.18... But when Markov actually counted, he found vowel-vowel pairs only show up 6% of the time, way less than if they were independent.”
Markov's demonstration meant that observing convergence in social statistics does not prove the underlying decisions were independent, so such statistics don't prove free will at all; independence isn't even necessary to do probability.
“"Thus, free will is not necessary to do probability." In fact, independence isn't even necessary to do probability.”
Markov built a two-state predictive machine using transition probabilities (e.g., 13% chance vowel-to-vowel, computed by dividing 0.06 by 0.43) and showed that running it produced a ratio converging to 43% vowels and 57% consonants—the exact split he had counted by hand—proving dependent events still follow the law of large numbers.
“after a while, it converges to a steady value, 43% vowels and 57% consonants, the exact split Markov had counted by hand. So Markov had built a dependent system, a literal chain of events, and he showed that it still followed the law of large numbers”
An overhand-style shuffle (the slow stacking motion shown) requires over 2,000 shuffles to randomize a deck, vastly more than the seven needed for a riffle shuffle.
“It's actually over 2,000... seven riffles or it doesn't count.”
Brin and Page modeled the web as a Markov chain by treating each link to a page as an endorsement (vote), where the more links a page sends out, the less valuable each individual vote becomes—measuring page importance by the long-run fraction of time a random surfer spends on each page.
“Brin and Page realized that each link to a page can be thought of as an endorsement. And the more links a page sends out, the less valuable each vote becomes. So what they realized is that we can model the web as a Markov chain.”
Running the neutron Markov chain on the ENIAC computer hundreds of times and tallying the multiplication factor k gives a statistical distribution: if k<1 the reaction dies, k=1 is self-sustaining, and k>1 grows exponentially into a bomb—approximating differential equations too hard to solve analytically.
“If you find that in most cases, k is less than one, the reaction dies down. If it's equal to one, there's a self-sustaining chain reaction... And if k is larger than one, the reaction grows exponentially and you've got a bomb.”
Yahoo's keyword search was easy to trick because you could repeat keywords hundreds of times hidden as white text on a white background; it had a notion of relevance but no notion of quality—which pages were actually better.
“Yahoo's keyword search was easy to trick. To get your page ranked highly, you could just repeat keywords hundreds of times, hidden with white text on a white background.”
Global warming involves a positive feedback loop: rising CO2 raises temperature, warmer atmosphere holds more water vapor (itself a powerful greenhouse gas), which raises temperature further—making the system hard to predict and hard to model with Markov chains.
“as the temperature increases, the atmosphere can hold more water vapor, which is an incredibly powerful greenhouse gas. And with more water vapor, the temperature increases further... So you get this positive feedback loop, which makes it hard to predict what's going to happen next.”
Von Neumann realized that because a neutron's behavior depends on where it is and what it has done before, you cannot sample independent random outcomes like Solitaire; you needed a Markov chain modeling a whole sequence of events where each step influences the next.
“A neutron's behavior depends on where it is and what it has done before. So you couldn't just sample random outcomes like in Solitaire... What von Neumann realized is that you needed a Markov chain.”
Solitaire with all 52 cards has 52 factorial (~8×10^67) possible games, making the probability of winning impossible to solve analytically; Ulam's insight was to play hundreds of games and count winnable ones to get a statistical approximation.
“the total number of possible games was 52 factorial, or about eight times 10 to 67. So solving this analytically was hopeless. But then Ulam had a flash of insight, what if I just play hundreds of games and count how many could be won?”
The memoryless property—that you can ignore a system's entire long history and look only at the current state to predict the next—is what makes Markov chains powerful, because it lets extremely complex systems be radically simplified while still yielding meaningful predictions.
“You can just look at the current state and forget about the rest, that makes these systems memoryless. And it's this memoryless property that makes Markov chains so powerful because it's what allows you to take these extremely complex systems and simplify them a lot to still make meaningful predictions.”
For a deck of 52 cards, riffle shuffling it seven times makes every arrangement of the deck about equally likely, so it is effectively random; modeling card shuffling as a Markov chain (each arrangement a state, each shuffle a step) yields this result.
“for a deck of 52 cards, if you riffle shuffle it seven times, then every arrangement of the deck is about equally likely, so it's basically random.”
Claude Shannon found that predicting text improves as you condition on more preceding units: using two prior letters yields partial words, while using whole words as predictors produces locally sensible four-word sequences—demonstrating that higher-order Markov models make better text predictions.
“Shannon learned that you can make better and better predictions about what the next word is going to be by taking into account more and more of the previous words.”
Creating 100 pages all linking to your own page won't beat PageRank, because although those links briefly make your page seem important, no other quality websites link to those spam pages, so over many steps their contribution to the random surfer's time distribution becomes negligible.
“none of the other websites link to them. So over many steps, their contributions don't matter. You might have many links, but they're not quality links, so they don't affect the algorithm.”
Modern large language models operate on tokens (letters, words, or punctuation) rather than just an alphabet, predicting the odds of the next token given a string of prior tokens—but unlike simple Markov chains they use 'attention' to decide which prior tokens matter, allowing context like 'blood' and 'mitochondria' to disambiguate that 'cell' means biology rather than a prison.
“unlike simple Markov chains, they also use something called attention, which tells the model what to pay attention to... the model can use previous context like blood and mitochondria to know the cell most likely refers to biology rather than a prison cell.”
When people shout out their guesses publicly rather than submitting them individually, the guesses become dependent (anchored by the first answer) and the average no longer converges to the true value but clusters around a biased amount.
“all the other people in the room are influenced by this value, and so, their guesses have become dependent. And now the average doesn't converge to the true value, but instead it clusters around a higher amount.”
Bernoulli only proved the law of large numbers worked for independent events, where one event doesn't influence the others; for 200 years probability relied on the assumption that independence is required to observe the law of large numbers.
“But Bernoulli only proved that it worked for independent events like a fair coin flip... And so, for 200 years, probability had relied on this key assumption, that you need independence to observe the law of large numbers.”
Nekrasov took Bernoulli's idea one step further, arguing that if you observe the law of large numbers in data, you can infer that the underlying events must be independent.
“He said, if you see the law of large numbers, you can infer that the underlying events must be independent.”
PageRank adds a damping factor where 85% of the time the random surfer follows a link and 15% of the time jumps to a random page, ensuring all parts of the web are explored without the surfer getting stuck in a loop in disconnected networks.
“we can set a rule that 85% of the time, our random server just follows a link like normal. But then for about 15% of the time, they just jump to a page at random. This damping factor makes sure that we explore all possible parts of the web without ever getting stuck.”
If LLM-generated text increasingly fills the internet and becomes training data for future models, the system enters a feedback loop that collapses to a dull, stable state repeating the same thing forever—a vulnerability of language models that, like other feedback systems, becomes hard to model with Markov chains.
“you come, in this case, to us, a very dull, stable state, it just says the same thing over and over and over again forever. The language models are vulnerable to this process.”
Markov chains enable probability with dependent events, which matters because in the real world almost everything is dependent on something else—weather, disease spread, and particle behavior all depend on prior or surrounding states.
“This should have been a huge breakthrough, because in the real world, almost everything is dependent on something else.”
Early search engines like Yahoo had no superior technology—they all ranked pages merely by how often a search term appeared on a page—so the battle for the top search engine would be decided by marketing spend and user acquisition rather than technical merit.
“None of the leading search engines at the time had any superior technology... They all just ranked pages by how often a search term appears on a given page. So the battle for the number one search engine would be decided by who could attract the most users, who could spend the most on marketing.”
Some critics argued better search was bad for business because if users find the right result on the first search, the engine loses the three or four additional searches (and ad impressions) it would have shown otherwise.
“if it takes them three or four chances, searches to get the right answer, then I have three or four chances to show ads, and if you get 'em the answer right away, I'm just gonna lose them. So... I don't see why better search is better.”
Page argued PageRank is 'a democracy that works' because treating all pages as equal would let anyone manufacture billions of pages to game ranking, whereas weighting by quality links prevents manipulation.
“I would say it actually is a democracy that works. If all pages were equal, anybody can manufacture as many pages as they want. I can set up a billion pages in my server tomorrow. We shouldn't treat them all as equal.”
The law of large numbers states that as you run more and more independent trials, the average outcome gets closer and closer to the expected value.
“This behavior that the average outcome gets closer and closer to the expected value as you run more and more independent trials is known as the law of large numbers.”
A nuclear chain reaction occurs when a neutron splits a U-235 nucleus, releasing two or three more neutrons; if on average those neutrons go on to split more than one other U-235 nucleus, you get a runaway chain reaction and a nuclear bomb.
“If, on average, those new neutrons go on to hit and split more than one other U-235 nucleus, you get a runaway chain reaction, so you have a nuclear bomb.”
Nekrasov argued that because social statistics like marriages, crime rates, and birth rates follow the law of large numbers, the decisions causing them must be independent—and therefore acts of free will, making free will scientifically measurable.
“So Nekrasov reasoned that because these statistics followed the law of large numbers, the decisions causing them must be independent. In other words, he argued that they must be acts of free will.”
The Monte Carlo method, named after the Monaco casino due to its random sampling and high stakes, spread quickly after the war—used by 1948 at Argonne for nuclear reactor designs—becoming a foundational simulation technique.
“the random sampling and high stakes reminded Ulam of the Monte Carlo Casino in Monaco, and the name stuck. The Monte Carlo method was born.”
Google's name originated from 'googol' (10 to the power of 100), reflecting the founders' ambition to index all internet pages, but they accidentally misspelled it when registering the domain.
“they thought of the largest number they could think of, 10 to the power of 100, a googol. But then when trying to register their domain, they accidentally misspelled it. And so, Google was born.”
The Markov chain that powers Google's trillion-dollar ranking algorithm only looks at the current state to predict what happens next.
“at the heart of this trillion dollar algorithm is a Markov chain, which only looks at the current state to predict what's going to happen next.”