Stephen Wolfram - Where the Computational Paradigm Leads (in Physics, Tech, AI, Biology, Math, ...)
What this covers
On Friday, October 18, 2024, Stephen Wolfram, Founder & CEO of Wolfram Research, gave a keynote talk titled "Where the Computational Paradigm Leads (in Physics, Tech, AI, Biology, Math, ...)" at the "Empowering Excellence: The Hertz Way" event, an evening for the Hertz community held at the American Academy of Arts and Sciences Cambridge, MA.
Source description (no synthesized summary yet).
Wolfram argues that simple computational rules can generate complex behaviors and that physics, biology, and even artificial intelligence all arise from fundamental principles of computation, with deep implications for how we understand the universe and design intelligent systems.
- Simple programs like Rule 30 demonstrate that computational irreducibility—the impossibility of predicting outcomes without running the computation—is a core phenomenon underlying physics and biology
- Discrete hypergraph rewriting at the fundamental level produces both general relativity and quantum mechanics, unifying three major 20th-century theories under computational origins
- Biological evolution and neural networks operate under the same principle of finding irreducible computational lumps that fit together, not by following simple narratives but by leveraging computational complexity
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Rule 30, a cellular automaton with a simple three-bit rule, produces patterns that appear completely random over many iterations despite starting from a single black cell, demonstrating that simple programs can generate complex and apparently random behavior.
“Rule 30 um it has that little uh program at the bottom but you started off from just one black cell and it makes this quite elaborate pattern keep going for a while it produces a pattern that for many practical purposes looks completely random you look at the center column in this pattern it seems completely random”
Cellular automata are simple programs where each cell's next state depends on its current state and neighboring cells according to fixed rules, and Wolfram discovered in 1981 by running all possible rules that many produce simple patterns, but Rule 30 produces elaborate, seemingly random behavior from just one initial black cell.
“back in 1981 or so uh I tried the obvious computer experiment which was just try running all possible rules of this kind and see what they do well the many of them do rather simple things but the big surprise and kind of my all-time favorite science Discovery is Rule 30 um it has that little uh program at the bottom but you started off from just one black cell and it makes this quite elaborate pattern”
Computational irreducibility is a consequence of simple rules generating complex behavior: for systems like Rule 30, there is no way to predict what happens after a billion steps without actually running the system for approximately a billion steps, preventing the kind of formula-based prediction that has defined exact science for 300 years.
“the question is well what's going to happen after a billion steps in this particular running this particular Rule and the surprising thing is that we have pretty good evidence that there's really no way to tell what will happen after a billion steps other than to run the thing for about a billion steps and see what happens”
Wolfram's life project has been building Wolfram Language, a computational language for describing the world in a computational way, analogous to how mathematical notation (plus signs, equal signs, etc.) was invented 500 years ago and allowed algebra and calculus to be created.
“the big effort of my life I suppose has been building wolam language and sort of the idea there...our mission now to be to try and create a kind of language a computational language for describing things that allows sort of computational X for all X to be created”
Computational irreducibility is the phenomenon wherein simple computational systems cannot be predicted to produce specific long-term outputs without actually running the computation for approximately that many steps—you cannot 'jump ahead' in prediction the way exact science has traditionally allowed.
“computational irreducibility and it has to do with the following thing in typically in in sort of one of the achievements of exact science is Let's Make a prediction for what will happen in a system well in this case you can ask well what's going to happen after a billion steps in this particular running this particular Rule and the surprising thing is that we have pretty good evidence that there's really no way to tell what will happen after a billion steps other than to run the thing for about a billion steps and see what happens”
In Wolfram's models, discrete hypergraph rewritings limit to the Einstein equations in the same way that discrete molecular motion limits to continuum fluid dynamics, suggesting that general relativity emerges from simple discrete computational rules.
“just as you can start as in those pictures that I was showing at the beginning with uh a bunch of discrete molecules bouncing around on a large scale a bunch of discrete molecules bouncing around sort of limit to a Continuum fluid kind of behavior so the question is what does a bunch of discrete hypergraph re writings limit to and it turns out this is something I kind of found out in the 1990s they limit to the Einstein equations”
Quantum mechanics is inevitable in these discrete hypergraph models: since there are many possible places where the hypergraph can be rewritten at each step, one must consider all possible rewriting paths simultaneously, creating a 'multi-way graph' of branching and merging possibilities.
“quantum mechanics is actually quite inevitable and the reason is that when you specify you know we've got this hypergraph are going to rewrite the hypergraph um the question is well there may be many different places where the hyp can be Rewritten how do you deal with that well the answer is that you can think about following all those possible Paths of rewriting you get what we call a multi-way graph”
The Feynman path integral, a fundamental mathematical formulation of quantum mechanics, is essentially the same as Einstein's equations, except the path integral is played out in 'branchial space' (the space of possible quantum branches) while the Einstein equations are played out in physical space.
“the F path integral which is kind kind of a mathematical foundation for for quantum mechanics the F path integral is basically the same as the Einstein equations except the path integral is played out in branchial space and the the Einstein equations are played out in physical space”
In the 20th century, the three big theories of physics—statistical mechanics, the second law of thermodynamics, and quantum mechanics—seemed independent, but they actually all arise from the same computational origin.
“the 20th century there were sort of three big theories in physics statistical mechanics second L of thermodynamics general relativity and quantum mechanics what seems to be the case is that all three of those theories actually come from the same origin the same kind of computational origin”
The second law of thermodynamics describes entropy increase because a system starts ordered and becomes disordered, but the process is time-reversible; what makes it irreversible in practice is that observers are computationally bounded and cannot decrypt the initial conditions from the final state.
“you have some some system of of of uh of particles that starts in an orderly configuration ends up producing uh apparent Randomness...the system is sort of encrypting its initial conditions and because we are computationally bounded we don't we are not able to decrypt those initial conditions”
When there is truly irreducible computation happening, neural networks and LLMs cannot reproduce it; their architecture and training process make them incapable of capturing genuine irreducibility.
“I don't think when there's truly kind of irreducible stuff going on I don't think it's it's it's just not in the nature of these things to be able to do that”
Neural networks fail at extrapolation: training a neural net to reproduce a sine curve in one region and then asking it to extrapolate beyond that region produces terrible results, even when the network size is increased.
“can you get a neural net to successfully reproduce um something that would otherwise come from physics equations and actually it doesn't work very well at all I mean this is an attempt with various neural Nets to reproduce a sign curve and after you know the region on the left it was it was trained for the region on the right is an extrapolation and it does horribly and it basically um and you know is even as you increase the size of the network it doesn't do any better”
When observers are embedded in the Ruliad, specific consequences follow: if observers are computationally bounded and believe they persist through time (maintain a coherent identity), these two assumptions alone are sufficient to derive the structure of general relativity and quantum mechanics.
“as soon as you start realizing that we as observers of this this thing are embedded within the ruad there start to be consequences...the two most important assumptions are that we are computationally bounded and that we believe we are persistent in time...those two assumptions alone are sufficient to give us the structure of general relativity and the structure of quantum mechanics”
Wolfram has spent years searching for algorithms that solve special functions to arbitrary precision anywhere in the complex plane by computing rational approximations, a technique that predates modern machine learning terminology but is essentially machine learning applied to mathematical problems.
“for years we've we've used so so like when when we first started building Mathematica um you know people I remember people said I wanted to evaluate SP all special functions you know hundreds of special functions to arbitary Precision anywhere in the complex plane and people said you're crazy you know they said by the end of the 1990s will have the integer order Bessel functions to quadruple precision and so what did we do well we built the system that just searched through uh sort of possible rational approximations”
The deepest question in Wolfram's physics is: what if the universe actually follows all possible computational rules simultaneously? This leads to the concept of the 'Ruliad'—the entangled limit of all possible computational processes, which may be the fundamental reality underlying physics.
“what if the universe actually followed all possible rules what would that be like...the end result of this is to think about this object that we call the ruad which is the sort of entangled limit of all possible computational processes”
Quantum mechanics is inevitable in hypergraph models because when rewriting a hypergraph, many different rewriting paths are possible at each step; following all possible paths creates a 'multi-way graph' where multiple threads of time coexist, representing the quantum superposition of states before measurement.
“the next big thing that you come up with in physics is quantum mechanics and it turns out in these models quantum mechanics is actually quite inevitable and the reason is that when you specify you know we've got this hypergraph are going to rewrite the hypergraph um the question is well there may be many different places where the hyp can be Rewritten how do you deal with that well the answer is that you can think about following all those possible Paths of rewriting you get what we call a multi-way graph”
Space is ultimately discrete rather than continuous; the universe consists of discrete elements related to each other, and everything—all particles and phenomena—are features of the structure of space itself.
“the kind of starting point of our efforts to understand sort of the machine code of physics is the the idea that space is ultimately discreet and that really all there is in the universe is the structure of space and everything that is uh kind of all particles and all those kinds of things are features of the structure of space”
The Ruliad—the entangled limit of all possible computational processes—can be thought of as the universe running all possible computational rules simultaneously, and making basic assumptions about observers (computational boundedness and persistence through time) is sufficient to derive the structure of both general relativity and quantum mechanics.
“you can kind of ask if you think about computational systems you think about there's a a little touring machine rule there's the touring machine running you can imagine a touring machine that has several possible rules you can imagine kind of building up this this sort of collection of what happens with touring machines with all possible rules and the end result of this is to think about this object that we call the ruad which is the sort of entangled limit of all possible computational processes”
The effective dimensionality of a hypergraph can be determined by measuring how the volume of an n-step neighborhood grows: if it grows as r^d, the space has approximately d-dimensional structure.
“you just start at some point in the hypergraph and you go at every step you kind of go one one unit away on the graph and you see how big is this what's the volume what's the number of nodes contained in this ball that has gone R steps if that grows like R to the D you say it's roughly D dimensional space”
The universe may be fundamentally discrete, composed of a hypergraph of discrete elements where space is the structure of relationships between these elements, and the universe evolves by rewriting this hypergraph according to simple rules.
“the kind of starting point of our efforts to understand sort of the machine code of physics is the the idea that space is ultimately discreet and that really all there is in the universe is the structure of space and everything that is uh kind of all particles and all those kinds of things are features of the structure of space”
In quantum mechanics, the technical question of how much effort (energy/computation) is required to 'knit together' multiple quantum branches into a single coherent experience is not addressed in traditional quantum formalism and represents a significant gap in the theory.
“because we humans believe in this sort of single thread of experience in order for us to make use of this quantum computer we have to have all those threads kind of knitted together to a single kind of conclusion and and that's sort of the the place where where um uh that that the real there's a big question which is not really addressed in quantum mechanics in its traditional formalism of sort of how much effort does it take to knit together all those threads of experience of of History to get this kind of so”
Modern machines built by humans are transitioning from an era where we understand them (post-industrial revolution) to an era where they may perform irreducible computation; there is a trade-off: we can either constrain machines to be computationally reducible (ensuring they only do what we want) or allow them to achieve maximum computational potential (accepting unpredictable behavior).
“the implications of computational reducibility I mean one one for sort of AI kinds of things is the following let's say you know we we now are in a situation where um the uh actually I could show something about that yeah so so um we're sort of now in a in a situation where uh you know we have had a period post industrial revolution when the machines we make we expect to understand um that is presumably coming to an end and the thing is we then have a choice do we want the things we make to be sort of doing their computational best which means they will be doing irreducible computation or do we want them to be constrained to do only the things that we want them to do so to speak”
Wolfram suspects that dark matter is not actually matter but a manifestation of the microscopic structure of spacetime—'space-time heat' arising from discrete degrees of freedom—analogous to how caloric fluid was replaced by microscopic molecular motion in explaining heat.
“Dark Matter um and I kind of see the following analogy back in the 1800s when people were thinking about heat they thought well heat flows so what do we know that flows oh it's a fluid so they invented caloric fluid turns out that wasn't the right theory of heat you know heat is microscopic motion of of of of of atoms and so on well I suspect Dark Matter may not be matter at all but instead something like space-time heat so to speak a feature of the kind of microscopic structure of of space”
Wolfram suspects that dimension fluctuations—regions where spacetime is not exactly three-dimensional but 3.01-dimensional—are effects that could test discrete spacetime models, raising interesting open physics problems like determining how photons propagate through non-integer-dimensional space.
“some of the effects that you see U are things like Dimension fluctuations the the universes does not have to be precisely three-dimensional so you can have regions of 3.01 dimensional space and so on it's an interesting physics problem what does Photon propagation look like through a region of 3.01 dimensional space uh not yet solved”
In branchial space—the space of possible quantum branches—gravity and geodesic deflection work identically to how they work in physical space, such that the Feynman path integral (quantum mechanics) and Einstein equations (gravity) are the same formalism applied to different spaces: branchial and physical respectively.
“in physical space in in this branchial space you can also ask things about deflection of gd6 and so on and well the sort of bottom line is it seems like the F path integral which is kind kind of a mathematical foundation for for quantum mechanics the F path integral is basically the same as the Einstein equations except the path integral is played out in branchial space and the the Einstein equations are played out in physical space”
Most physicists at the beginning of the 20th century believed space would turn out to be discrete, but lacked tools to formalize this; Einstein himself believed discreteness was likely but said 'we don't have the tools to see how that works yet', which after 100 years is now possible.
“beginning of the 20th century most physicists believed that space was discrete as well but nobody could make that work and people like Einstein would say you know it will turn out to be discreet but we don't have the tools to see how that works yet well 100 years later we do have some of those tools”
The effective dimensionality of a discrete hypergraph can be determined by measuring how the volume (number of nodes) grows with graph distance: if volume grows like R^D, the space is approximately D-dimensional, but the actual dimension emerges from the structure and is not predetermined.
“you just start at some point in the hypergraph and you go at every step you kind of go one one unit away on the graph and you see how big is this what's the volume what's the number of nodes contained in this ball that has gone R steps if that grows like R to the D you say it's roughly D dimensional space”
In the hypergraph model, time is not a separate dimension but rather the progressive rewriting of the hypergraph structure itself—the passage of time corresponds to the progressive irreducible computation associated with hypergraph evolution.
“when it comes to we talk about the structure of space being defined by this hyper graph time is just the progressive rewriting of the hypergraph the progressive kind of irreducible computation associated with the rewriting of the hypergraph”
In classical physics definite things happen, but in quantum physics there are many different threads of possibility that eventually 'knit together' during measurement, representing a difference in how the universe's branching structure is unified through observation.
“in classical physics sort of definite things happen in quantum physics you have kind of the view that there are these many different threads of possibility which then eventually we kind of knit together when we when we try and make a measurement or something”
Large language models (LLMs) trained on the protein data bank can successfully predict protein structure when dealing with previously observed peptide sequences that fit known patterns, but fail on novel sequences, suggesting they are finding known reducible structures rather than discovering how to fold arbitrary proteins.
“what was happening was you know you take the protein Data Bank you do multiple sequence alignment you can find that the protein you're looking for you know pieces of it fit things that were already known and then kind of the machine learning part of it was kind of fitting together those pieces that were already known when you have a completely abono you know here's a random sequence of peptides how will it fold up it is it is I think it's completely unclear whether that produces a sensible result and my guess is that it doesn't”
Three-dimensionality of perceived space is likely not a fundamental feature but an observer-specific property—just as general relativity and quantum mechanics arise from basic observer assumptions (boundedness and persistence), the perception of 3D space arises from more specific observer properties not yet identified.
“other things like the fact that we perceive space as threedimensional are features that have to do with more detailed properties of us as observers haven't figured out and and they'll probably be very obvious once we see them”
Computational irreducibility makes existence meaningful: if everything were computationally reducible and we could know all future outcomes in advance, there would be nothing achieved by the passage of time; irreducibility ensures that the passage of time accomplishes something rigidly new.
“computational irreducibility is sort of what makes existence meaningful in the following sense if everything we did was just we could know what was going to happen you know the answer is going to be 42 or whatever there would be nothing achieved by the passage of time computational irreducibility kind of shows that something is sort of rigidly achieved by the passage of time”
Computational irreducibility is not merely a limitation of science but something humans live with constantly in nature; we have found viable paths for existing in the natural world despite the irreducible computations constantly occurring around us.
“it's kind of like you know that we are pretty used to living around computational irreducibility because we live around nature which is full of that kind of thing and we've sort of found these particular paths for existing in in the natural world where we can where we we can sort of Happily exist even though there are these sort of irreducible computations that that are happening”
Neural nets versus finite element methods: in scientific computing, neural networks still do not outperform established numerical methods like finite element methods, even after years of research.
“lots of things we've done experiments with for ages like neural Nets versus finite element methods still not really working well”
Dark matter may not be matter at all but a manifestation of microscopic spacetime structure—analogous to how heat was once thought to be a fluid (caloric) but turned out to be molecular motion—making dark matter evidence of space-time's granular nature rather than unknown particles.
“my favorite is actually dark matter um and I kind of see the following analogy back in the 1800s when people were thinking about heat they thought well heat flows so what do we know that flows oh it's a fluid so they invented caloric fluid turns out that wasn't the right theory of heat you know heat is microscopic motion of of of of of atoms and so on well I suspect Dark Matter may not be matter at all but instead something like space-time heat so to speak a feature of the kind of microscopic structure of of space”
Wolfram has begun exploring the possibility of a formal foundational theory of medicine by considering organisms evolved for particular fitness criteria and then asking: if the organism is perturbed or 'poked,' can it still achieve its purpose? This allows classification of possible diseases and investigation of how perturbations can be remedied.
“one can find a sort of a foundational theory for things like medicine which one would not think there might be a foundational theory for but once you have this idea that you can have these kind of organisms that are evolved for a purpose you can say well what happens if poke the organism and perturb it in some way...you can start to say as you poke it can you classify the the possible diseases that can happen”
In the late 1970s, Wolfram became interested in how complexity arises in the world and hypothesized that simple programs and rules could serve as the most general medium for making models of things, rather than mathematical equations which didn't work well.
“I got interested in sort of the general question of how comp complexity arises in in the world and uh uh that got me interested in kind of well what how can you make models of things like that and I tried using sort of mathematical equations things like that didn't work very well I started thinking you know what what is the most General kind of medium that we can use to make models of things”
It is not obvious that the configuration of discrete space atoms at one moment of time will be the same entity as at the next moment, yet we assume observer identity persists; this assumption is a fundamental feature of how we experience the world.
“when we think about sort of us being made of these atoms of space that are in these different configurations and so on there's it's not at all obvious that the configuration of of atoms of space at one moment of time will be will give us sort of the same us at a subsequent moment of time”
When an LLM generates Wolfram Language code to solve an open-ended query, the output is readable and interpretable by humans at a level that would be impossible with low-level programming languages, enabling human-AI collaboration where the human can refine and build on the AI's output.
“I think this sort of this collaboration between the human and the AI of you say some vague thing it produces something that is a piece of of precise computational language uh chances are you know I can more or less read this and I think this sort of this collaboration between the human and the AI...this wouldn't work with a low-level programming language because you got a big blob of code which most people couldn't read kind of the idea is this is sort of high enough level that a human can read it”
Wolfram identifies a key trade-off in AI design: machines can either be constrained to do only what we want them to do (computationally reducible, limited, predictable) or allowed to do their full computational best (computationally irreducible, capable of unexpected behaviors). Modern AI is moving toward the latter because irreducibility enables capability.
“do we want the things we make to be sort of doing their computational best which means they will be doing irreducible computation or do we want them to be constrained to do only the things that we want them to do so to speak you kind of have this trade-off you can force the thing to be computationally reducible so you know what all the pieces do and you can know that it will only do the things you want it to do or you can allow it to do what to sort of achieve as much as it can computationally”
The fundamental question about AI is: when do you need actual computation, and when can you just use an LLM? There are tasks that require true computation and tasks that can be solved by LLM pattern matching, and determining which is which is the core technology question.
“the question is uh when do you end up using computation when do you end up being able to sort of just go through the layers of an llm to get a result”
Modern neural networks and machine learning work best on human-like tasks where humans have intuition, but perform poorly on physics problems without human conceptual input, because neural networks essentially find lumps of irreducible computation that happen to fit together rather than discovering underlying physical laws.
“the thing about neuron Nets when are neural Nets going to work well they work on a lot of things which are human-like tasks and they probably distinguish cats from dogs in kind of the same way that humans do because they work kind kind of like humans work now when you're doing some problem in physics about proteins there is no kind of human angle to that so it's it's much less clear what's what's going to happen”
Wolfram Language and computational language design enables humans to specify complex computational tasks at a high level, which is more powerful than having AI generate low-level code because humans can read and understand the computational language descriptions.
“this is sort of an example of uh this is an example of something somewhat interesting which is we've given it some vague thing to do it's produced something in precise computational language uh chances are you know I can more or less read this and I think this sort of this collaboration between the human and the AI of you say some vague thing it produces something that is a piece of of precise computational language I mean this wouldn't work with a low-level programming language because you got a big blob of code which most people couldn't read kind of the idea is this is sort of high enough level that a human can read it a human can can take those kind of building blocks and go from there”
Mathematical equations were not effective for modeling complexity, leading Wolfram to consider simple programs and cellular automata as more general foundations for models, based on the insight that specifying rules for something determines what it will do.
“I tried using sort of mathematical equations things like that didn't work very well I started thinking you know what what is the most General kind of medium that we can use to make models of things and started thinking about sort of if you just specify rules for something what do those rules specify that it should do and thinking about simple programs as kind of the basis for models of things so I started studying these things get called cellular autometer”
Quantum computer noise may be a sign of underlying discrete spacetime structure—specifically, the 'maximum entanglement speed' (analog of speed of light in branchial space)—making quantum computing engineering noise potentially evidence for the discrete model if the right patterns are observed.
“in quantum computers I I have the suspicion that a bunch of the noise that is seen in quantum computers is actually a sign I'm kind of hoping that the engineer of quantum computers has been done so well that if one plots out the right things one will actually see kind of a noise floor that is associated with a thing that in our models we call the maximum entanglement speed which is the analog of the speed of light in branchial space”
Computation-augmented generation (CAG) is a technology approach where LLMs generate code that is then executed by actual computation engines, as opposed to retrieval-augmented generation (RAG) which retrieves data; CAG is necessary when tasks require irreducible computation.
“the sort of emerging kind of Technology uh connection is what we're calling U computation augmented generation which is kind of the analog of retrieval augmented generation but computation augmented generation means you're an llm and you're producing in output and you're basically using computation to to fuel the output you're generating”
Boolean neural networks (neural networks built from discrete Boolean cells with two possible states) are simpler to visualize and analyze than traditional continuous neural networks; Wolfram created an example of a minimal Boolean neural network that evolves cellular automaton rules to achieve a specific purpose like 'live for 50 steps then die.'
“this is an example of kind of a very minimized kind of Boolean IED neural network in which you you have uh you have just two possible rules that you can run at every cell and then you're asking can you can you make the neural net can you kind of evolve a neural net this is a very simplified neural net that will achieve some particular purpose so in this particular case the purpose being achieved was live for 50 steps and then die out”
The problem with AI automatically writing code is not that AI cannot generate code, but that specifying what you actually want the code to do is the fundamental bottleneck; Wolfram has spent decades developing notation for precisely specifying computational specifications.
“the main problem is what do you want the code to do you have to have some specification for that and you know that's what I've been trying to build for a long time is a good notation for specifying what you actually want the code to do”
Neural networks are analogous to stone walls built by fitting irregular rocks together to fill gaps: just as a mason finds rocks that fit gaps left by previous rocks, machine learning finds lumps of irreducible computation that happen to fit together to achieve a desired output.
“if you're building a wall there are sort of a couple of approaches you can take you can engineer it by building a bunch of bricks and then carefully arranging the bricks you get a wall that way another possibility is you can see a bunch of rocks lying around on the ground and you can build a stone wall by finding sort of which rock fits into which kind of Gap that's left and you can successfully build a wall...I think machine learning is basically doing something like that it's taking lumps of irreducible computation and finding ones that happen to fit”
Minimal formal theories of biological disease can be constructed by treating organisms as evolved systems, perturbing them, and categorizing what kinds of failures result—analogous to how a disease classification system (like the ICD) works for biological organisms and computer security categories work for computer systems.
“one thing I was looking at very recently is the following thing can one find a sort of a a foundational theory for things like medicine which one would not think there might be a foundational theory for but once you have this idea that you can have these kind of organisms that are evolved for a purpose you can say well what happens if poke the organism and perturb it in some way and it then doesn't you know does it still achieve its purpose does it still live a long life or whatever else and what you can start to do is you can start to say as you poke it can you classify the the possible diseases that can happen some it will recover from some it will not”
Computation-augmented generation—combining language models with actual computation engines rather than replacing computation with pure neural network inference—is the appropriate technological approach because there are problems that fundamentally require computation and cannot be solved by pattern-matching alone.
“the sort of emerging kind of Technology uh connection is what we're calling U computation augmented generation which is kind of the analog of retrieval augmented generation but computation augmented generation means you're an llm and you're producing in output and you're basically using computation to to fuel the output you're generating and there are things where you kind of have to use actual computation and there are cases where you can just use the llm”
One of the lessons Wolfram learned early on was that tools exist but people don't always use them, and those who do use them gain great leverage.
“one of the things I learned at that time was uh you know people there are tools but people don't always use them if you do use them you have great leverage”
We are accustomed to living amid computational irreducibility because nature is full of it, and we have found particular paths for existing within natural systems that contain irreducible computations without being harmed by them.
“we are pretty used to living around computational irreducibility because we live around nature which is full of that kind of thing and we've sort of found these particular paths for existing in in the natural world where we can where we we can sort of Happily exist even though there are these sort of irreducible computations that that are happening”
The complete space of all possible evolutionary pathways in simple cellular automata can be mapped and visualized, revealing analogies to causal graphs in relativity where different fitness criteria correspond to different reference frames in spacetime.
“this model is simple enough that you can kind of map out the uh uh the complete structure of all possible Evolution paths and so you can kind of see on one side it sort of has one idea about how to grow on the other side it has a different idea about how to grow”
The computational language mission is to create what people did 500 years ago with mathematical notation (streamlining verbal mathematics into symbolic form), but now for computation: creating tools that allow 'computational X for all X' to be invented and practiced.
“I kind of see our mission now to be to try and create a kind of language a computational language for describing things that allows sort of computational X for all X to be created and we've spent the last I don't know 38 years or so building this this”
The cellular automata evolution model is simple enough that the complete structure of all possible evolution paths can be mapped, revealing which paths were taken: one side has organisms with one growth idea, the other side has organisms with different growth ideas, and a megamapper structure appeared in the middle.
“because this is kind of a map of all possible Evolution paths that could that could happen in this in this particular uh space of organisms and the thing that I I realized there's a there's another piece of it I guess that's a sort of a megap forner thing that showed up in the middle uh um the U the thing that I sort of find interesting about this is this is this is a map of all possible Evolution paths”
In Wolfram's cellular automata model of evolution, genotypes are cellular automata rules, and phenotypes are the spatial patterns they generate; by introducing point mutations to rules and selecting for behaviors like 'live as long as possible but not infinitely,' one can evolve sequences of different phenotypes over successive generations.
“the rules are on the left you start the thing off from just one red cell on the right it it runs for some number of steps and then dies out now you imagine making point mutations to those underlying rules and you ask can I uh can I make a series of point mutations so that I will achieve a particular purpose so for example one thing you might try to do is say live as long as possible but not for an infinite time”
The progression of fitness values in Wolfram's cellular automata evolution model closely resembles a machine learning loss curve inverted: red dots represent failed attempts, jumps upward represent breakthroughs when more successful organisms are discovered, revealing a similar optimization structure to neural network training.
“this is kind of a fitness uh this is looking at the progression of fitnesses looks very much like much like a loss curve in machine learning except it's turned upside down...the red dots are kind of the attempts that were made to find successful sort of organisms and the jumps up where finally an idea was had a breakthrough was made”
Neural networks trained on protein folding data work well for regular secondary structures like alpha helices but fail on complex, 'big glob mess' structures, suggesting that neural networks can capture computationally reducible patterns but struggle with irreducible ones.
“there's a decent example um what you see is you know you have an alpha Helix that's behaving in a fairly simple way it will do a pretty good job of reproducing it when it's a big glob mess it doesn't do a terribly good job of reproducing it”
Current approaches of directly replacing algorithmic code with neural networks (e.g., using neural networks instead of finite element methods for solving differential equations) are not promising because algorithms were already discovered through searches and neural networks add little over optimized algorithmic approaches.
“the things that we use for I don't know solving differential equations and so on those were found by Searchers the the algorithms were found by searches so it's not you know the things that we use for...solving differential equations and so on those were found by Searchers”
Wolfram notes that the diversity of cellular automaton organisms evolved for different purposes is striking, with small changes in underlying rules producing completely different shapes and functions, analogous to how changing a few amino acids in a protein (like an antibody) can produce completely different shapes and functions.
“how diverse all these Critters are and that it's kind of like you know you're making an antibody or something and you know you're just changing a few you know a few peptides and the thing ends up in a completely different shape with a particular completely different function and this is sort of a minimal model of something like that”
The second law of thermodynamics arises because systems encrypt their initial conditions, and computationally bounded observers cannot decrypt them, making ordered initial states appear random.
“what what it's done and effectively what happens is that the system is sort of encrypting its initial conditions and because we are computationally bounded we don't we are not able to decrypt those initial conditions that the we're not able to decrypt the state that we get to find out that it came from something simple”
When evaluating proteins, neural networks benefit from human annotation of features (like beta sheets), which injects human reasoning back into what appears to be pure machine learning.
“when you're looking at a protein there are there are sort of features of that protein that we humans tend to pick out like oh there's a beta sheet here and in so far as you're saying look this neural net did really well it picked out these same kinds of features that's again something where you're injecting kind of human angle into into what's going on”
Sexual reproduction added to the cellular automata evolution model makes it more complicated with endless parameters, but the end results are not substantially different from asexual evolution.
“for example sexual reproduction I was surprised that it really gives one nothing very different the model is more complicated there are endless little sort of parameters to introduce and so on but the end results are really no different”
Machine learning can be used to search for and discover algorithms (like special function approximations), which Wolfram and team did successfully for decades before modern neural networks—a comparable technology to current AI-based algorithm search.
“when when we first started building Mathematica um you know people I remember people said I wanted to evaluate SP all special functions you know hundreds of special functions to arbitary Precision anywhere in the complex plane and people said you're crazy you know they said by the end of the 1990s will have the integer order Bessel functions to quadruple precision and so what did we do well we built the system that just searched through uh sort of possible rational approximations to these functions we did what today would be called machine learning”
In 1985, Wolfram attempted to model biological evolution using cellular automata by starting with a rule and mutating it to achieve particular behaviors, but the approach was unsuccessful at the time, partly because he hadn't known about machine learning.
“I had started doing this in 1985 uh I was curious about whether one could have sort of a minimal model of biological evolution using things like cellular autometer and at the time I had tried sort of uh starting off with some cellular automatan Rule and saying can I mutate that rule to get certain kinds of behavior and I hadn't managed to make that work but I didn't know about machine learning at that time”
The book cover illustration was made in 1960 at Lawrence Livermore Lab by Bernie Alder, who was recruited by Edward Teller in 1955 to work on high-density materials, and the computer used to create it was called the LARC (Livermore Advanced Research Computer).
“the picture was made in 1960 at Lawrence lmore lab um by uh a chap called Bernie aler who had been recruited to Lawrence lmore in 1955 by Edward Teller to work on um kind of high density materials”
Around 1981, Wolfram conducted a computational experiment of running all possible cellular automaton rules to see what behaviors they produced, which led to the discovery of Rule 30.
“well back in 1981 or so uh I tried the obvious computer experiment which was just try running all possible rules of this kind and see what they do well the many of them do rather simple things but the big surprise and kind of my all-time favorite science Discovery is Rule 30”
Bernie Alder, recruited to Lawrence Livermore in 1955 by Edward Teller to work on high-density materials, created the 1960 illustration of molecules bouncing to illustrate the second law of thermodynamics that appeared on the cover of the statistical mechanics book that inspired Wolfram's childhood computational interest.
“the picture was made in 1960 at Lawrence lmore lab um by uh a chap called Bernie aler who had been recruited to Lawrence lmore in 1955 by Edward Teller to work on um kind of high density materials and so on”
In the early 1990s, Wolfram made initial progress on understanding the universe as arising from simple rules, and then in 2019 made significantly more progress as a result of a technical advance, suggesting that the project has been an ongoing lifelong effort with periodic breakthroughs.
“I thought about that u in the early 1990s I made some progress on that but then right in 2019 uh sort of as a result of first a quite technical uh Advance uh I kind of was able to make a lot more progress on that”
In June 1972, Wolfram purchased a book about statistical physics with an illustration of molecules bouncing around to demonstrate the second law of thermodynamics, which inspired his first major computational effort: attempting to simulate that picture on a desk-sized computer.
“sometime in uh June of 1972 I bought myself a book about statistical physics and I got very interested in this and I really like the cover of the book which is an illustration of kind of the uh molecules idealized molecules bouncing around illustrating second law of Thermodynamics”
After studying particle physics and cosmology in the late 1970s (described as the golden age of particle physics and quantum field theory), Wolfram became interested in the general question of how complexity arises in the world.
“after studying particle physics and cosmology and so on in the in the 1970s late 1970s which was a great time because that was it was sort of the golden age of of particle physics and Quantum field Theory and so on I got interested in sort of the general question of how comp complexity arises in in the world”
By the end of the 1800s, the scientific question of whether matter is discrete or continuous had been resolved: molecules existed and matter was discrete, though the discreteness of light and space remained unresolved.
“if you go back to Antiquity people were arguing forever about whether the universe is discrete or continuous and that argument continued through the 1800s by the end of the 1800s people had nailed it molecules existed matter was discreet likewise you can think of light as discreet at that time”
Computers on which thermodynamic simulations were run at Lawrence Livermore (like the LARC) were about the size of a desk and represented state-of-the-art technology in 1960.
“and actually the computer I have a picture here of the computer on which that book uh cover was made in those days the the desk came with a computer that was a thing called the Lark the Livermore Advanced research computer”
Wolfram's first attempt to simulate molecular dynamics on a desk-sized computer was unsuccessful initially, but he later learned he had produced something more interesting than the original picture.
“I did it on a on this computer here was about the size of a desk um wasn't successful actually um um I didn't manage to reproduce that picture as I leared a decade later I actually had produced something more interesting”
Tools exist but people don't always use them; those who do use computational tools to handle mechanical tasks (like mathematical calculations) gain great leverage over those who don't.
“you know people there are tools but people don't always use them if you do use them you have great leverage”
In the 1990s, Wolfram found that discrete hypergraph rewriting systems limit to Einstein's equations, and in 2019, as a result of a technical advance, he was able to make much more progress on the physics project.
“I thought about that u in the early 1990s I made some progress on that but then right in 2019 uh sort of as a result of first a quite technical uh Advance uh I kind of was able to make a lot more progress on that it's sort of been an interesting thing that's kind of the result of my sort of Life trajectory alternating between doing basic science and doing technology development”
The speaker wrote a new book last year (green-covered, resembling the original 1960 statistical physics book) that finally provides an understanding of how the second law of thermodynamics works, fulfilling an intellectual journey begun 50 years earlier when he first encountered that original book.
“I was kind of happy 50 years after I I got that book about um which had the nice picture of of um uh from about statistical mechanics I I wrote this book last year which kind of is is green and looks a bit like the the previous book which I think is finally an understanding of how the second law works”
Wolfram's Herz Foundation connection is through his wife Ailis, who was also a Herz Fellow, and they have been married for approximately 30 years (as of the time of this talk).
“I first heard about the Herz Foundation probably 45 years ago and I there were all kinds of interesting people that I knew and I happened to hear oh they had some Herz Fellowship thing I hadn't really quite put all of it together until I think sometime after Alis and I got married 30 years ago now I kind of leared she was also one of these hurts fellow people”
Boolean neural networks—minimal neural networks composed of discrete cells with only two possible states at each cell—can be evolved to solve specific computational tasks, suggesting that deep learning need not rely on continuous valued weights or complex architectures.
“this is an example of kind of a very minimized kind of Boolean IED neural network in which you you have uh you have just two possible rules that you can run at every cell and then you're asking can you can you make the neural net can you kind of evolve a neural net this is a very simplified neural net that will achieve some particular purpose so in this particular case the purpose being achieved was live for 50 steps and then die out”
The speaker realized this evolution-relativity analogy yesterday and finds it interesting because it provides a more theoretical approach to biological evolution by connecting it to reference frame selection.
“that was a thing I just realized yesterday so the the um uh and I think that's that's sort of interesting because it gives one kind of a a a handle on a more theoretical approach to things like biological evolution”
The speaker asked a question about endosymbiosis—whether the evolutionary system could be used to predict how successful two systems would be at generating symbiosis, and whether environmental constraints could be modeled—but had not yet explored these questions.
“I haven't looked at anything to do with symbiosis this is a what what's interesting about this is a very simple model and I hadn't realized that a model this simple could give any of the kinds of features that one is interested in biological evolution”
A questioner at the Hertz Foundation event asked whether endosymbiosis and symbiosis between systems with independent evolutionary rules could be predicted and modeled with Wolfram's cellular automaton tools, whether environmental constraints could be incorporated, and what would happen if two evolved organisms were combined.
“so one of the most important inventions of life which I I also myself research is the process of endosymbiosis and in general um symbiosis between two different systems that have their own evolutionary rules I'm wondering if this tool um could be used to predict what how successful two systems would be in generating some type of symbiosis and then is there a way to implement different constraints from the environment to assess that”
A questioner described themselves as working on using AI to design molecules validated in lab experiments, and expressed frustration that computational irreducibility suggests the process is 'luck' rather than aligned with known physics, asking whether the molecules discovered align with known physics or whether it's purely irreducible complexity.
“I work in a space where we're trying to use AI to design molecules that we can then validate in the lab and I'm really frustrated by the suggestion that this is completely irreducible because like most scientists that I work with want some physical explanation for why those molecules were correct”