fabian s. klinkeThe Math That Predicts (Almost) Anything
Source: The Strange Math That Predicts (Almost) Anything, Veritasium, 32:32, uploaded 2025-07-25, playlist index 210.
The video opens with four questions that seem to belong to different fields: how many times a deck needs shuffling before it becomes random, how much uranium a nuclear bomb needs, how a system predicts the next word in a sentence, and how Google finds the page a person wants. Veritasium traces each answer to Markov chains, a form of probability that grew from a mathematical feud in Russia.
Independence and free will
In 1905, socialist groups across Russia rose against the Tsar. The political split reached into universities, and the video places two mathematicians on opposite sides of it. Pavel Nekrasov, a religious and influential probability theorist who supported the Tsar, argued that mathematics could say something about free will and the will of God. Andrey Markov, an atheist who had little patience for what he regarded as loose reasoning, attacked Nekrasov’s work as an abuse of mathematics.
Their dispute concerned the law of large numbers. Flip a fair coin ten times and six heads with four tails is unremarkable, even though it differs from the expected half. Keep flipping and the ratio moves towards 50/50. Jacob Bernoulli proved this behaviour in 1713 for independent events, where one trial does not influence another.
The assumption becomes visible when the trials depend on one another. If people estimate the value of an object in private, each estimate can count as a separate observation. Ask them to call out their guesses in a room and the first person can pull the rest towards an inflated figure. The average may settle down, although it settles around the shared influence rather than the object’s true value.
Nekrasov accepted the usual direction of the argument: independence produces convergence. He also proposed the reverse. If social statistics converge in the manner described by the law of large numbers, he thought the decisions behind them must be independent. He pointed to Belgian marriage figures from 1841 to 1845, whose yearly average stayed close to 29,000, and to similar patterns in crime and birth rates. Since people choose whether to marry, commit crimes, or have children, Nekrasov treated the stable averages as evidence that those choices came from free will. The video presents this as his mathematical route from regular statistics to a scientific account of free will.
Markov wanted a dependent system that still converged. Text gave him one. A letter’s status as a vowel or consonant depends strongly on the letter before it, so he took the first 20,000 letters of Alexander Pushkin’s Eugene Onegin, removed spaces and punctuation, and counted the results. Vowels made up 43 percent of the string and consonants 57 percent. If letters were independent, a vowel followed by a vowel would occur about 18 percent of the time. Markov found that pair only 6 percent of the time, with the other pair frequencies also differing from the independent case.
He turned those counts into a small prediction machine. The two states were vowel and consonant. From a vowel, the chance of moving to another vowel came from the observed vowel-vowel frequency divided by the overall vowel frequency, which gives about 13 percent. The chance of moving to a consonant was the remainder. He filled in the same transition probabilities for consonants, then generated one random number after another to choose the next state.
The simulated sequence jumps around at first. As it grows, its vowel and consonant ratio approaches the 43/57 split Markov measured in the poem. The chain has dependence at every step, yet its average converges. That example breaks Nekrasov’s inference: convergence in a statistic cannot prove that the events producing it are independent, and the social figures cannot prove free will.
The paper did not become a public triumph. The video quotes Markov saying that he cared about pure analysis and regarded questions of practical application with indifference. The mathematical object he built acquired a much larger life than its creator expected because real systems contain dependence everywhere. Tomorrow’s weather depends on today’s conditions, disease spread depends on current infections, and a particle’s behaviour depends on its surroundings. A Markov chain gives those systems a way to carry probability through a sequence of dependent states.
Neutrons, Solitaire, and Monte Carlo
The next application begins at Los Alamos. On 16 July 1945, the United States detonated the first nuclear bomb, a six-kilogram plutonium device whose explosion matched nearly 25,000 tons of TNT. The Manhattan Project mathematician Stanislaw Ulam kept working after the war on a much harder question: how neutrons move through a fissile core and whether their interactions sustain a chain reaction. The video gives the physical picture at a high level. A neutron can scatter, leave or become absorbed, or trigger fission that releases more neutrons. The probabilities depend on the neutron’s position, speed, energy, and the configuration of the material.
In January 1946, encephalitis left Ulam recovering in bed. He played Solitaire and began wondering what fraction of randomly dealt games could be won. A deck has 52 factorial possible arrangements, which the video gives as roughly 8 × 10⁶⁷. Enumerating them was impractical. Ulam could instead play hundreds of games, count the wins, and use the fraction as a statistical estimate.
The analogy became useful when Ulam returned to Los Alamos. A Solitaire deal is independent of the previous game. A neutron carries the influence of its position and earlier interactions, so the nuclear problem needs a chain in which each step changes the next step’s probabilities. John von Neumann recognised that Markov’s construction supplied the missing form.
The simplified model starts with one neutron moving through the core. It may scatter and continue, disappear from the chain, or cause fission and create new neutron chains. The transition probabilities vary with the neutron and the material, so the model must carry enough of the current state to choose the next event. The ENIAC generated starting conditions, followed the chain, and measured the average number of new neutrons produced in each run. The video calls that average the multiplication factor, k.
When k stays below one, the chain dies down. At one, it sustains itself. Above one, it grows. Repeating the simulation hundreds of times produces a distribution of possible outcomes, which lets researchers estimate behaviour that would be difficult to calculate by solving the underlying differential equations directly. Ulam’s uncle was a gambler, and the combination of random sampling and high stakes led Ulam to the Monte Carlo Casino in Monaco. The name remained.
The method spread from weapons research to reactor design. The video quotes Ulam’s later surprise that a few marks on a blackboard could change human affairs. That line also describes the structure of the episode: a counterexample made for an argument about free will becomes a tool for simulating physical systems whose exact state space is too large to enumerate.
The web as a random walk
The same construction later helped solve a less explosive problem. When the public internet grew in the 1990s, thousands of pages appeared each day and search engines had to decide which ones deserved attention. Yahoo, founded by Jerry Yang and David Filo in 1994, ranked pages largely by the frequency of a search term. The system could be gamed by repeating keywords in hidden text.
Masayoshi Son saw that the leading search engines had similar technology and offered Yahoo’s founders far more money than they had asked for. The video uses the episode to show how a search business could be won through marketing when the ranking method itself offered little quality signal. A better method needed to measure whether a page deserved a high position, rather than only whether it mentioned the right word.
Larry Page and Sergey Brin borrowed an idea from library books. A card covered in due-date stamps suggested that many readers had used the book. A link to a web page could play a similar role as an endorsement, with the value of a link reduced when the linking page sent out many other links. They modelled the web as a Markov chain in which each page is a state and each link is a transition.
The video demonstrates the system with four pages called Amy, Ben, Chris, and Dan. A reader starts on a random page and follows links. Over many steps, the proportion of time spent on each page settles into a ranking. Ben receives the highest score, followed by Amy, Dan, and Chris. A page owner could try to manufacture votes by creating a hundred linked pages, although those pages have little weight when the rest of the web does not link to them.
The web has disconnected regions and loops, so a random surfer could remain trapped. Page and Brin solved that with a damping rule. The surfer follows a link 85 percent of the time and jumps to a random page for the remaining 15 percent. This makes the chain explore the web instead of settling inside one isolated loop. They called the method PageRank, a name that refers to both the behaviour of pages and Larry Page.
PageRank gave Google better results than Yahoo’s keyword system, often placing the wanted site first. The founders initially called their search engine BackRub because it analysed backlinks. Their larger ambition required a larger name, so they chose googol, the number 10¹⁰⁰, and misspelled it when registering the domain. Google launched in 1998 and overtook Yahoo within four years. The video places a Markov chain beneath the search engine’s later scale and influence, whilst leaving the historical story in the form of a documentary account rather than a sourced company history.
Prediction, attention, and feedback
Claude Shannon returned to Markov’s text experiment in the 1940s. He generated text by predicting letters from earlier letters, then tried pairs of letters and full words. The output becomes more plausible as the model uses more context, although it still produces sentences whose local fragments make sense whilst the whole remains incoherent. A sequence such as “attack on an English writer” can sound grammatical even when the surrounding sentence has lost its subject.
Modern predictive text works with tokens. A token can be a letter, a word, or punctuation, and a model estimates the probability of the next token from the preceding context. Large language models add attention, which lets the system assign different weight to different parts of that context. In “the structure of the cell”, words such as “blood” and “mitochondria” push the prediction towards biology rather than a prison cell.
The video then names a weakness in a system that trains on its own output. If generated text enters the internet and becomes training data for later models, the feedback loop can pull the language towards a dull, stable state that repeats itself. The speaker does not attach a named study or measured forecast to this warning in the captions, so it belongs in the note as a model risk described by the video rather than as an established prediction about every language model.
Feedback also limits the Markov simplification. A model can use the present state to predict the next state when that state contains the information the process needs. A system with feedback keeps changing the conditions that shape its future. The video uses warming air and water vapour as an example: higher temperature permits more vapour, and the extra greenhouse gas raises the temperature again. A simple chain may still describe parts of such a process, although the feedback makes the next state harder to estimate.
That is what “memoryless” means here. A Markov chain does not claim that the system has no history. It claims that, given a suitable current state, the past can be left out of the next-step calculation. The current state may need to include position, energy, context, or a long enough token sequence. Choosing that state is the modelling work. The video quotes a paper’s compact formulation: problem-solving often consists of cooking up an appropriate Markov chain.
The shuffle at the end
The closing example returns to the opening question. Each deck arrangement can serve as a state, and each shuffle can serve as a transition. For a 52-card deck, the video says that seven riffle shuffles make every arrangement about equally likely, which is close enough to call the deck random in practice. The claim concerns that particular shuffle model and means approximate mixing, rather than a literal guarantee that every possible arrangement has equal probability after the seventh movement.
The hand shuffle shown in the video produces a different result. Its answer is over 2,000 repetitions. The number matters less than the route to it. A common action becomes a large state-space problem, and a Markov chain turns the problem into a question about how quickly a distribution loses its dependence on its starting arrangement.
Limits
The historical narrative, figures, quotations, and technical explanations in this note follow Veritasium’s video and its English captions. The description credits Brian Hayes, David Aldous, Geoff Engelstein, Jeffrey Rosenthal, Jimmy He, Mark Priestley, Michael Choi, Peter Norvig, Sam Power, and Thomas Haigh for expertise and contributions, plus Madeline Johnson for fact-checking, and it links a separate references page. The captions do not identify which contributor supports each claim, and I have not independently checked the historical episodes, the nuclear figures, the PageRank history, or the language-model warning against those underlying sources.
The durable mathematical claim is narrower than the title. Markov chains make useful predictions for many dependent systems when the current state contains enough information to determine the next-step probabilities. The choice of state controls what the model can forget, and feedback can make that choice unstable. The method reaches from a poem to a nuclear simulation, a search ranking, a text predictor, and a card shuffle because those cases share a structure, not because one small formula predicts everything.