What this covers
Stephen Wolfram hosts a live and unscripted Ask Me Anything about the history of science and technology for all ages. Find the playlist of Q&A's here: https://wolfr.am/youtube-sw-qa Originally livestreamed at: https://twitch.tv/stephen_wolfram If you missed the original livestream of this episode, feel free to submit a question you would like Stephen to answer in a future Q&A livestream here: https://wolfr.am/12cczmv5J 00:00 Start stream 00:57 SW starts talking 1:11 What would you say is the most important human-designed algorithm of all time? 15:32 Historically, who has led the trends in science, practitioners or academics? 34:35 Did Richard Feynman really think that "philosophy is baloney"? Did you ever discuss non-physics subjects? 45:29 If simulation becomes sufficiently good in the future, will it cause experimental scientists to be out of a job? 52:58 How did we go about solving the goat problem? 53:14 According to the history of science, what might be the ratio of the number of minor paradigm shifts to the number of major paradigm shifts? 55:25 What was the fifth class of cellular automata that almost was, which you mentioned in your personal history paper? 58:48 Has an idea like the ruliad existed before, or is this a novel object? 1:02:13 Neural networks show that combining two seemingly unrelated fields of research can produce great results, but our academic and business cultures are focused more on specialization. Your thoughts? 1:10:32 What would a modern analog computer look like today? Follow us on our official social media channels. Twitter: https://twitter.com/WolframResearch/ Facebook: https://www.facebook.com/wolframresearch/ Instagram: https://www.instagram.com/wolframresearch/ LinkedIn: https://www.linkedin.com/company/wolfram-research/ Stephen Wolfram's Twitter: https://twitter.com/stephen_wolfram/ Contribute to the official Wolfram Community: https://community.wolfram.com/ Stay up-to-date on the latest interest at Wolfram Research through our blog: https://blog.wolfram.com/ Follow Stephen Wolfram's life, interests, and what makes him tick on his blog: https://writings.stephenwolfram.com/
Source description (no synthesized summary yet).
Wolfram argues that major scientific advances emerge from interdisciplinary thinking and unusual backgrounds rather than well-institutionalized mainstream research, and that understanding fundamental mechanisms—whether algorithmic, computational, or conceptual—is essential to scientific progress.
- Institutionalization creates constraints that inhibit innovation; big advances rarely come from heavily structured, mainstream areas
- Cross-disciplinary approaches and tool-building drive discovery as much as tool-use; the separation between practitioners and academics obscures genuine innovation
- Computational irreducibility means simulation cannot replace experimental science; some phenomena must be observed empirically
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Statistics originated in the idea of reducing individual-level complexity (specific people, deaths, births) to aggregate numbers (population, mortality rate), answering questions about collectives rather than individuals; this reductive approach remains the dominant paradigm despite the availability of raw sensor data that could support richer, more direct analysis.
“I think, you know, statistics was all about, you know, its original idea it was sort of understanding the properties of a state like a state in a government kind of state in terms of a certain set of aggregate numbers. Like what's the overall population? Yes, we can say there's this person with that name and that name and that name that all live in the city. But, now we say, what's the aggregate? What's the statistics of this? How many people all together are there here? How many people all together died in this year or whatever else? Um and this sort of reduction of the complete to individual numbers, that's been sort of a story of statistics for a long time.”
The linear feedback shift register algorithm designed by Sol Golomb is the most-used human-designed algorithm in history, deployed roughly a quadrillion to octillion times across CDMA telephony, GPS, WiFi, and computer display data transmission because its sequences are maximally non-repetitive and non-correlated, allowing multiple signals to share the same radio frequencies without interference.
“I think the answer is the algorithm that generates the linear feedback shift register sequences designed by a person I knew named Sol Golomb... that's a what that's used for is when you have you're trying to generate a sequence of bits that you don't want to repeat and you don't want sub blocks of that sequence of repeat of of bits to repeat.”
Linear programming and the simplex algorithm, invented in the 1940s (originally for operations research and logistics in World War II), solve optimization problems by treating linear inequalities as hyperplanes that slice space, with the solution always located at a corner, and the simplex algorithm is an exponential algorithm in general until sub-exponential algorithms were discovered in the 1980s.
“Other algorithms that are famous are things like the linear algebra linear programming algorithms like the simplex algorithms invented by in the 1940s, I guess, um for taking a collection of inequalities and saying, "What will be the place where these inequalities are are all satisfied, or what will be the place where where this quantity is maximized subject to these inequalities?"”
Modern large-scale academic enterprise with structured organization dates from post-World War II; before that, academia was primarily for training clergy, lawyers, and other professions, with science being added later; this contextualizes who counts as an 'academic' versus a 'practitioner' and complicates these categories historically.
“first of all, modern academia is is fairly new, um sort of the the pattern of the kind of large-scale academic enterprise with its very uh structured uh organization, it probably dates, well, dates from basically post-World War II.”
The startup company landscape mirrors institutionalized science: once venture capital funding and growth metrics become defined norms, startups become a 'well-defined production line' with much less possibility for dramatic innovation, parallel to how mature scientific fields limit breakthroughs.
“I mean, I I think, you know, people sometimes say, oh gosh, you know, there's no innovation anymore, look, all these startup companies are all doing the same kinds of things. Well, that's because, in a sense, by the time you define your universe of discourse to be startup companies that go through venture capital and do this and do that, well, you've got yourself to a situation that's very much like kind of the normal phase of science, where the the sort of the highly innovative, we don't really know how this fits together type thing, but there's a sort of big leap that can happen there, has already happened, and you've got to the point where sort of it's a well-defined production line, and much less sort of dramatic leaps forward can happen.”
Most developed scientific and academic areas have value systems that privilege tool-use or tool creation but exclude or undervalue certain contributions (e.g., tool-building is not valued for tenure), creating misaligned incentives where important work goes unrewarded.
“it tends to be the case that the kind of um, uh, the kind of value system of a well-developed area like most areas of academia tends to be one where it's it's homed in on oh, the the use of the tools is valuable or the tools are valuable or something else is valuable. And for example, it can be the case that people are doing something that um, where they're in an area where it's mostly the write the paper that records the results and so on and oh, you're building tools and nobody cares about that. You can't you know, get tenure building tool tools and so on. Which is unfortunate because that is actually, you know, some part of where a lot of the value comes is in things like the building of tools and there can be an area where the sort of the value system is just kind of excluded those kinds of um, those kinds of contributions.”
Major scientific advances rarely emerge from the vast institutionalized mainstream of well-defined work; instead, they typically come from unusual positions, people with funky or cross-disciplinary backgrounds, or those outside the mainstream field, which means mature, highly institutionalized areas of science tend to be poor generators of innovation.
“when you ask the question, where do significant advances come from, um where they don't come from usually is the vast institutionalized ma- mass of kind of well-defined work. That's a rare place for for big advances come from, although there are exceptions.”
The Charles Babbage/Ada Lovelace example from the 1840s exemplifies the division between tool-makers and scientific thinkers: Babbage was focused on building mechanical computers for mathematical tables, while Lovelace recognized the more abstract principle of computation, demonstrating that tool-building and theoretical insight are distinct intellectual activities.
“We had a very early example of something like this in the Charles Babbage Ada Lovelace story from the 1840s, where, you know, Charles Babbage was very much a I'm going to make mathematical tables, I'm going to build these mechanical computers to do that, and Ada Lovelace, you know, who themed herself more of a sort of a poetess of science, so to speak, was like, well, what's the real overall point of this? And then started realizing things about, well, there's this sort of, in a sense, more abstract idea of computation.”
As engineering matures (e.g., airplane wing design via wind tunnels), computational fluid dynamics and machine learning can often replace physical testing for many design variations; however, novel regimes (e.g., a space plane entering the atmosphere at Mach 20) exceed the bounds of the model and require new experimental approaches.
“There was a time when the only thing you could do was use a wind tunnel and just see but then computational fluid dynamics got better to the point where it was possible to test many kinds of designs using computational fluid dynamics. I think in modern times you know there's sort of machine learning that lets you interpolate between the results of of computational fluid dynamics and experiments and so on and you can say well you know if I add this little blob to the wing is it going to what kind of effect is it going to have? Well you can figure that out by simulation.”
Universal computation principle explains why a single neural network can be applied across different domains (language generation, protein folding, image processing)—not surprising given that Turing machines can compute arbitrary functions; however, neural nets deployed as single-pass systems cannot implement arbitrary computation length, crushing computational irreducibility.
“You could say, well, gosh, golly, isn't it surprising that this one computer can do all these different kinds of things? But that's the idea of universal computation. That's the idea you can have that single Turing machine that is universal, that can do all these different kinds of computations. So that's one point, that to say we've got the system that is uh sort of can do all these different things is is perhaps less surprising than it might be.”
GPUs were primarily built by practitioners for general-purpose graphics computing purposes, not specifically for machine learning, yet became foundational infrastructure for the deep learning revolution of the 2010s.
“GPUs as a thing that have driven sort of a bunch of machine learning methodology. And uh you know, GPUs were very strong much built by practitioners, so to speak, um for purposes that weren't specific to um the use case”
Any simulation not based on the absolute bedrock of physics (the lowest-level structure of the universe) uses a model that approximates the world, capturing some features and discarding others; models can miss unknown effects, which is why experimental verification is necessary even for phenomena within well-understood domains.
“Now I think it's and also whenever you do simulation that isn't on the bedrock so to speak right down at the level of the atoms of space or whatever else the really the lowest level structure of the universe whenever you do that you're always doing it on the basis of a model. And a model is something which approximates the universe approximates the world capturing some features and discarding others.”
The discovery of the cosmic microwave background by Arno Penzias and Robert Wilson in 1964 is an example of a scientific advance that emerged from practitioners outside the academic mainstream working at Bell Labs on satellite communications, who happened to notice background radio noise and connected it to Big Bang theory.
“the discovery of the cosmic microwave background by by um, uh, uh, Penzias and and Arno um, in 1964 or so where you know, they were working at Bell Labs. Um, I think they had physics backgrounds, I think. Um, yeah, but that must be the case. Yes. Um, but they were working on satellite communications and they noticed this background uh, radio noise and they knew about the the Big Bang theory of the universe and they sort of were able to put two and two together and make a science advance.”
When methodological advances (whether conceptual like quantum field theory or instrumental like X-rays) are made in science, there is typically a short period (5-10 years) of intense 'low-hanging fruit' discovery, followed by a long 'cruise period' of 50-100 years of incremental progress before another major breakout occurs.
“the general course of the history of science tends to be there is some new methodology invented. That methodology might be a conceptual methodology, or it might be an instrumentation methodology, but once that's been invented, there are then lots of things that can be done with it, and usually over a fairly short period of time, 5, 10 years, something like that, there's sort of lots of low-hanging fruit that gets picked.”
Kary Mullis, inventor of PCR (polymerase chain reaction), is an example of a wild character with an unusual background who worked for a biotech company, proposed a 'funky crazy idea' that everyone said wouldn't work, but it did—and while PCR is more of a tooling advance than a pure science discovery, it has been critical to enabling subsequent scientific advances.
“there are I don't know why I'm thinking of Kary Mullis, the inventor of of PCR. Um, uh, who uh, I don't exactly know his background. He's a kind of a a wild character at least when I knew him. Um, but uh, it's kind of a that was a case where it was sort of a a more of a practitioner driven, you know, this is a funky crazy idea and everybody told him it couldn't possibly work and he was working for a biotech company and uh, well, it turns out it did work and it turns out it's been very important um, in now is that really, you know, is that sort of a science advance is that a tooling advance? It's PCR has been critical to lots of science advances um, but you know, as a as a thing itself it was more perhaps a tooling advance.”
If we knew the fundamental theory of physics and could work upward from foundational principles, experimental science would theoretically be obsolete because physics would reduce to mathematics; however, the phenomenon of computational irreducibility prevents this because knowing the underlying rules does not allow one to jump ahead and predict outcomes without tracing through irreducible computational steps.
“if we know the fundamental theory of physics and we can work up from that and get to everything in physics then yes the experimental scientist would be out of a job cuz we've reduced physics to a branch of mathematics so to speak. But there's a big catch which is this phenomenon of computationally irreducibility that just knowing the underlying rules doesn't immediately let you jump ahead and say so this is how things are going to this is how everything is going to work.”
Telescope and microscope exemplify how instrumental methodology creation launches entire scientific fields, with rapid initial progress in astronomy and biology respectively driven by the availability of these tools.
“one saw that early on with the telescope, uh early on with the microscope. You know, telescope sort of launched astronomy in a bunch of areas of physics. Microscope launched various areas of biology”
Primitive recursion, a concept from early 20th-century mathematical logic, describes operations where you know in advance how many times you will loop; general recursion (which Gödel introduced for his incompleteness theorem work) describes operations where you continue until some criterion is satisfied, as exemplified by the Euclidean algorithm.
“The first sort of do things a a certain number of times, that's reminiscent of what people call primitive recursion back in the beginning of the 1900s when people were trying to describe, well, what are the possible operations that you can do that make up something like a mathematical function?”
Analog computers (mechanical, electronic, photonic) represent an alternative paradigm where computation is performed on continuous variables represented by physical states rather than bitified (discretized) representations, exemplified by slide rules, Kelvin tide-predicting machines, and potentially modern photonic quantum computers.
“by in a sense, when you say analog computer, what you're saying is we're not going to go through uh trying hard to make the idealization that data is represented in bits. We're just going to have the data represented by, I don't know, a electromagnetic field representing some light at this place in a hologram or something like this. Um and and not say specifically I want to bitify the thing um and then see what happens to those bits. I mean, even, you know, any computer once taking those bits and one's doing sort of quotes analog physics to go from one configuration of bits to another, but you're concentrating on oh, and the thing the state of the system is represented in terms of bits. An analog computer, I think, sort of the defining feature is you never represent it in terms of the state of bits. It's just the state of some system that could include something you imagine as a continuous variable.”
Naive schoolbook multiplication of n-digit numbers requires n² operations, but computational tricks can reduce this to approximately n log log log n operations, and such optimized multiplication algorithms are implemented in every modern CPU and arithmetic logic unit.
“for multiplication, you would naively think but sort of standard schoolbook method method of doing multiplication of numbers with n digits each is every digit has to be sort of uh compared with every other digit every digit in the other number and you would take n squared operations to do that. But in fact, it's possible to rearrange that computation so that you don't have to do all those all those kind of combinations. And in fact, one knows one can get n log log log something n uh number of steps to do multiplication although the the constant in front is quite big.”
Without deliberate leadership intervention, institutional structures spontaneously replicate themselves and inhibit bridge-building across domains; the structural defaults work against cross-disciplinary innovation.
“it's not going to spontaneously happen. It's going to spontaneously the structures that exist are going to replicate themselves, and the let's make this bridge to this other structure, it's very unlikely to happen.”
Feynman believed that 'peace of mind is a prerequisite to creative work' and advocated cutting oneself off from the world to avoid stress from practical concerns—a philosophy that valued disconnection from worldly distractions for enabling creative output.
“peace of mind is a prerequisite to creative work. And he always he took the point of view that that it was important to sort of cut yourself off enough from the world that you just didn't have to worry about practical you know dealing with this or that thing which would cause at least for him you know some amount of stress and that sort of disconnecting from those kinds of things was crucial to being able to do creative work.”
Experimental science is undergoing transformation due to high-volume sensor data collection; modern experiments generate gigabytes of data per second rather than single-number outputs, requiring new frameworks beyond statistics and machine learning for comparing theories to data and extracting meaning from massive datasets.
“I think experimental scientists won't be out of a job. I think what is true is that there is a transformation of experimental science as a result of of sensor data and the idea, you know, back in the day, it's like we do an experiment and we measure one number and that's what our experiment is about. In today's world, it's like let's actually measure, you know, we we're getting a gigabyte a second of data from this experiment and it's a kind of a gusher of data and both in terms of the getting of the data with sensors and the well, what do we do with that data?”
Analog computers are limited for non-linear systems because the mapping of non-linear differential equations to linear electronic circuits is difficult; this limitation was a primary reason analog computers were abandoned for general-purpose digital computing.
“Now, an electric circuit like that is fundamentally linear in its operation, so that means that if you've got a heavily non-linear system, it's hard to map it to that linear differential equation that can be implemented just using um kind of uh standard electronics and so on.”
Steepest descent (gradient descent) and Newton's algorithm, used in modern backpropagation, are based on principles described by Newton himself for finding roots of equations, with Newton's method iteratively converging toward roots or extrema by following the downward slope given by calculus derivatives.
“the steepest descent uh kind of um algorithm where you've got some you know, you've got some function you're trying to find its minimum... you continually try to follow the the downward direction as found by calculus... That type of algorithm again, very early in history, Newton uh I think really did describe that what's now called Newton's algorithm for finding, for example, roots of equations or or square roots or whatever else where you're successively iterating to get closer and closer to the root”
Kelvin's tidal computing machine mechanically added rotating wheels with different periods to compute tides, analogous to Ptolemaic epicycles; the machine exploited the fact that tides are sums of multiple periodic components.
“Kelvin, for example, in the 1870s, 1880s developed an analog computer for working out the tides. Where he would have essentially it's like a clockwork machine because the tides have these different components... he was doing something where you're kind of rotating one wheel and that makes another wheel go this amount. You're adding up the rotations of all these wheels to compute the tides... a little bit like the Ptolemaic system in astronomy”
The 'rule yard' is the ultimate limit of all possible computational processes in an entangled state, an object that appears obvious to study but, to Wolfram's knowledge, has never been studied before his work beginning approximately 18 months prior to the interview.
“the rule yard is this thing that I started describing a year and a half ago now um that is this kind of ultimate limit of all possible sort of ultimate entangled limit of all possible computational processes. So far as I know, it's a really obvious object to study and it's never been studied before so far as I know.”
Cross-disciplinary and boundary-spanning research is unequivocally important for scientific and technological progress, yet academia's structure of specialized silos makes it difficult to organize and support such work, creating a persistent tension between institutional organization and innovation.
“the question of whether sort of the cross-fertilization of different areas is important to progress in in academic and business, and so on, absolutely. That's almost without exception. The the really energetic stuff is happening at the boundaries between existing areas, where you apply an idea from this area to that area, and so on.”
Leadership becomes essential for enabling cross-silo innovation: a leader must deliberately create a new initiative that bridges established silos (A and B) but isn't housed in either, and must champion it despite its lack of natural institutional home.
“that's often where sort of leadership is important, is just defining we are going to have this new thing. It will not emerge from silo A or silo B. It's got to be something that um uh somebody, some crazy person has to come and say we're going to we're going to invent this thing, and it's going to take things from silo A and silo B, but it isn't in either of those silos.”
Automated theorem proving tools built by practitioners have enabled only one surprising scientific discovery to date: Wolfram's 2000 discovery of the minimal axiom system for Boolean algebra; most other applications simply verify known results, illustrating the gap between tool-building and the scientific use of tools.
“automated theorem proving is something that has been long studied. There are many practitioners who who kind of do that, but if you ask, "What are the things that have been discovered by automated theorem proving that we didn't already know as opposed to things where you've kind of filled in sort of yes, we can automate it automatically prove this thing um just as we could have done it by hand?" Well, I think the only example still to date of a surprising result discovered by automated theorem proving was the thing that I discovered in the year 2000 about finding the minimal axiom system for Boolean algebra, which I found by using automated theorem proving tools that had been sort of built by practitioners, but they were not the ones who were thinking about kind of the basic science of what can be done from this.”
The understanding that computation is cheap (the universe can waste vast amounts of computation by running parallel branches of all possible states) represents a paradigm shift from earlier views where computational steps were imagined as expensive and thus rare; Wolfram's acceptance of computation as cheap enabled the rule yard concept.
“what has happened is, and this has happened a lot in our physics project, I've understood slowly that just uh computation is can be thought of as cheap in a sense. You know, there was a time when people imagined that these computational steps, they're very expensive, there's just one path, we just look at that, and so on. But the idea that the universe can just be an absolute unbelievable waster of computation. The idea that there can be all these different cases that are all just running in parallel, and everything's happening, and and most and what we sense is just this unbelievably tiny slice of all the things that are happening.”
Feynman had great calculational ability and would often perform complex calculations, then work backward to find simple, intuitive explanations without going through the complicated steps—leading people to think his intuitions were original insights when they actually followed from the calculations, though his intuition about what calculations would work was genuinely strong.
“one of his greatest skills was was calculational ability and you know, he used to do this thing which which was always amusing where he would do this big complicated calculation and he would think oh, everybody can do that, which isn't true at all. But then he would go and and think given that he got the answer, he would say well, what's an easy way that I can derive this without going through all these complicated calculational steps and he would come up with this kind of great intuitive explanation which wasn't where it originally came from and then everybody would say, oh wow, you know, he comes up with these amazing intuition explanations. His intuition was good but but um, uh, and his intuition about what calculations would work was very good.”
Powerful analogies work because one domain has been well-developed (over ~100 years of research with much known) while another is fresh and underdeveloped; mature concepts can be imported into the less-developed area, allowing rapid 'physicalization' and advancement comparable to the mature domain.
“But once but then in the other area, it's very fresh. It hasn't been worked out, but you can now import the things from the well-developed area into the less developed area and oh gosh, you can then take the less developed area and suddenly sort of physicalize it, bring it up to a a level of development, let's say, that's comparable to what's happened in physics, for example. It's something I was doing recently with meta-mathematics. Physicalization of meta-mathematics. I've sort of taking ideas from physics and importing them into the much less well-developed area of meta-mathematics.”
Wolfram himself succeeded in doing unconventional science because he entered academia early, achieved a secure position in his early twenties (or earlier), and was thus able to do work outside mainstream fields while still earning a living—a privilege he recognizes most academics lack.
“people like myself who, you know, I'm managed to kind of get through the academic sort of uh system pretty quickly, and I was in a pretty sort of secure academic kind of role, and I I could do kind of uh things by the time I was, well, in my early twenties, I was perfectly, perhaps even earlier than that, I was I was perfectly in a position to kind of do kind of uh uh science that wasn't quite in the main line of any existing area, and it was okay. I could, you know, uh I could make a living even though I was doing that.”
Feynman had a self-image of being poor at administrative and people-management work, claiming he was not good at activities involving coordination and making things happen; however, Wolfram believes this self-assessment was not entirely accurate.
“he had a in terms of sort of life philosophy I think he had this kind of he had a certain self-image about what he was good at what he wasn't good at which I don't think was always correct... one of the things he he always used to claim was that you know anything administrative anything that involved sort of people and getting things to happen and so on is like he's not good at that. I don't think that was entirely true”
Wolfram has deliberately inhabited both the tool-making world (building Wolfram Language) and the tool-using world (applying those tools for discovery), and has found that discoveries from tool-use feed back into tool development, creating a productive feedback loop that he considers valuable.
“I myself happen to have sort of consciously both been in the tool making world and in the using of the tools kind of world and that's been something I've very intentionally done um, of you know, I spend a bunch of time making tools like Wolfram language and then I also spend a bunch of time trying to use those tools and it turns out often the things that I discover from using those tools actually feed back into the development of new tools and that's pretty neat when that happens.”
Feynman's philosophy of life principle is best characterized as 'if you put in the effort, you can understand anything'—a principle he valued highly, viewing the opposite ('trust me, I'm an expert') with strong negativity.
“one that I very much ascribe to myself is uh, uh, sort of the uh, well, he would characterize it as there are no experts. I would characterize it more as if you put in the effort, you can understand anything. And uh, you know, I think he was he was always very negative and I think it's maybe the same kind of thing that I was talking about about kind of grounding anything you think about in steps you can understand. He was very negative about the situation where, you know, the expert will come in and say the answer is X. Um, and just trust me, I'm an expert, you know, the answer is X. He really uh, I think as I say, my my version of it would be more, you know, if you put in the effort, um, you can uh, kind of um, you can understand the steps.”
Feynman disliked thinking that 'floats off' into word-batting without grounding in concrete steps or understanding, preferring ideas anchored in calculable foundations where one could trace the reasoning through to justified conclusions.
“I think Dick Feynman always liked sort of sort of thinking that was grounded. And whenever things started floating off into the we're just batting words around and we don't really know what the words mean, he didn't like that at all.”
Feynman never built large groups of collaborators around himself, instead operating as a solo scientist with occasional collaborators who were primarily social companions or tool-builders (running programs, writing code) rather than strict scientific collaborators.
“He never really did that. He was always kind of a solo operative with occasional collaborators who were I would say more social collaborators than they were sort of strictly scientific collaborators or sometimes sometimes collaborators who would you know do tool type things of of running programs or writing programs or whatever else. But I would say more the criterion for for for Feynman of collaborators was I would say probably more more sort of social and scientific and and not the kind of thing where he wanted to build up a big group and he had to manage the group and so on. He didn't want that at all.”
Telescope-grinding was originally a practitioner skill (lens makers), and though Galileo improved telescopes and made astronomical discoveries, the core technology of grinding lenses and making telescopes came from practitioners, not from the academics who later used them.
“telescopes were built by, you know, originally by sort of lens-grinding practitioners, so to speak... the use case that Galileo put telescopes to... Galileo improved the telescope, but the basic idea of of grinding lenses and making a telescope um came from from practitioners of that of that area. Um uh although those practitioners aren't necessarily the people who make the scientific advance”
TRIZ (Soviet innovation methodology) attempted to mechanically generate innovations by crossing ideas from different fields, presupposing that innovation can be systematized; Wolfram believes this approach does not work well because innovation cannot be structured as rigidly as TRIZ suggests.
“there's this thing uh was called TRIZ, which was a Soviet-era uh concept... the concept was, you kind of take an idea from one area, idea from another area, you kind of cross them over, and that will generate deterministically a new innovative idea. I don't think it works particularly well... you you can you can imagine you can get a little bit too structured about structured about how to make unstructured innovations”
Historically, many scholars occupied intermediate positions: some worked in universities but also served as court philosophers, diplomats, or practitioners; Leibniz exemplifies this, being employed as a backroom diplomat and genealogist rather than a traditional academic, and this 'court philosopher' role, though rare now, has modern analogues in Silicon Valley and AI philosophy circles.
“there was a was a complicated thing of were you sort of an academic working in the kind of semi-monastic university, or were you somebody who was at the court of King So-and-So, or Queen So-and-So, um and you were sort of the the the court mathematician or something.”
Feynman tended to be interested in and comfortable around unusual, non-academic folk (artists, business people, people with different viewpoints) and found this stimulating, whereas most academics feel most comfortable with other academics at conferences.
“he had a definite tendency for being interested in the unusual folk and you know he just wasn't very interested in the whole well it's part of the main line thing and it's part of the the kind of um you know we're we're going to put another another step in this in this long chain so to speak. He and he tended to like hanging out with kind of the unusual folk the artists the sometimes business people sometimes the people you know with with different kinds of points of view I think he always he always liked that. And he always found that I think stimulating and it was um and so for for other academics it was like you know that that almost didn't compute because there tends to be a you know academics do tend to you know feel most comfortable and hang out most with academics at conferences or whatever else.”
PCR can be classified as both a scientific advance (enabling biological research) and a tooling advance (creating new experimental capability), illustrating the ambiguity in how institutions value different types of innovation.
“is that really, you know, is that sort of a science advance is that a tooling advance? It's PCR has been critical to lots of science advances um, but you know, as a as a thing itself it was more perhaps a tooling advance”
Wolfram contemplated a potential fifth class of cellular automata (Class 5) to distinguish between localized structures with no background activity versus localized structures with significant background activity interacting with them, but abandoned this distinction after encountering too many corner cases and ambiguous time-scale phenomena.
“But, I had thought that one could distinguish cases where uh what would happen is it breaks into localized structures where the structures really are separated from a case where there is sort of a background of sort of crisscrossing type things or something like that, a background of activity where things don't really separate, but they sort of separate. Um and I had thought you could distinguish those two classes and that was kind of the class five idea”
Slide rules (using logarithmic scales) have been used since the 1600s as analog computers, where physical distance on scales represents multiplication through logarithmic properties—a mechanical analog computing paradigm that remained useful until electronic calculators arrived in the early 1970s.
“slide rules, for example, from the 1600s. You know, slide rules for those who aren't as ancient as as I am and and used them at one time. Uh you know, slide rules were used in in until the early 1970s when calculators kind of took over. But a slide rule is this thing where you you have this scale, it's usually a logarithmic scale and you you have this sliding part, you have this cursor that reads between the scales and you use the fact that if you literally move a slide out in some direction and the slide is marked in logarithmic increments, then when you look at you know, the the fixed piece and the slide that was moved out a certain direction distance, you can multiply numbers just by looking at sort of how the the uh the logarithmic units on the slide sort of add to the logarithmic units on the on the on the main part of the slide rule.”
The Antikythera device (~100 AD) is an ancient analog computer with gearing mechanisms that can be moved to any angle; while it has teeth (giving it a somewhat digital appearance), it is fundamentally analog because gear ratios determine rotational rates continuously.
“presumably the Antikythera device from around 0 AD um was uh is this is this computer that has has gearing and I suppose one could think about it as sort of an analog computer, too, where where the analog uh where the gears that can be moved to any uh the the the the wheels can be moved to any angle. It's not digitalized, although it looks a bit digital cuz it has, you know, it has teeth on the gears that are used to mesh the gears together and I suppose also used to be to have the feature that you know, if you have two gears next to each other, the the the ratio of the number of teeth will be the ratio of the of the uh rates of rotation of the different gears.”
Backpropagation is the primary neural network training algorithm deployed today, performing trillions to quadrillions of operations in typical training runs, though Wolfram speculates this may not be the ultimate endpoint for neural net training methodology.
“the backpropagation algorithm, for example, um that's certainly been a an algorithm that's been used uh that's been applied many times, you know, in a typical neural net training these days. It's trillions to quadrillions of those kinds of operations that are being done... My guess is that's actually not the ultimate sort of end point of how neural net training should be done, but that's the way that it's done right now”
The mathematical theory of finite fields provides the foundation for deriving which patterns of tap positions in a shift register will produce maximal-length linear feedback shift register sequences, a theoretical insight developed by Sol Golomb in the 1950s.
“It turns out that some math that comes from uh theory of finite fields gives you information about what will be the maximal length linear feedback shift register sequences. That's what Sol Golomb worked out in the 1950s”
The Euclidean algorithm for computing greatest common divisors, described in Euclid's Elements, is an iterative algorithm where you repeatedly subtract numbers until reaching nothing, and it is one of the earliest formally described algorithms in history.
“There are sort of iterative algorithms where you're doing something, you keep on doing it until for example, something no longer changes like the Euclidean algorithm for working out GCDs. That is sort of described in Euclid um is an algorithm of that type where you keep on subtracting numbers and so on and until until you get down to there being nothing left.”
Divide-and-conquer recursive algorithms break a problem into subproblems, solve them independently, and recombine results; the quicksort algorithm (invented by Tony Hoare in the 1950s) exemplifies this approach by sorting a list in n log n operations rather than the naive n² comparisons.
“Another huge class of algorithms are the kind of divide and conquer truly recursive algorithms where you take a thing and you break it down and you consider sort of uh sub cases of that, and you build it back up again. Famous example of that is the quick sort algorithm due to Tony Hoare from the 1950s, I guess, which was an early computer-implemented algorithm.”
Pathfinding algorithms like Dijkstra's algorithm and A* (invented in the 1950s or 1960s) solve the problem of finding a sequence of moves from one configuration to another, such as rearranging tiles in a plane or navigating a graph.
“There are also lots of algorithms that are kind of pathfinding algorithms where it's like there is a way to get from here to there. There is a way to um sort of arrange these tiles so that you can uh put them in the plane. There is a way to find a series of moves that you can make that get you from one configuration to another. Those kinds of algorithms, and there were um uh the Dijkstra's algorithm, the A* algorithm, these were invented, I think, in the 1950s or 1960s um that are algorithms for doing this kind of pathfinding.”
Cellular automata can be classified into four categories: Class 1 (dies out), Class 2 (reaches steady state), Class 3 (produces randomness forever), and Class 4 (intermediate: produces localized structures with complicated interactions), where Class 4 is most computationally sophisticated and visibly complex.
“I was like, what do I do now? I'm doing natural science. How do I classify the behavior that I'm seeing? And it was actually rather obvious. I mean, a little bit like, you know, if you say, I've got this material substance and it's a solid, liquid or a gas. Which is it? You know, classify it. There might be corner cases, but basically you can say it's a solid, it's a liquid, it's a gas. Similarly with cellular automata, it was quite easy to say class one, the ones where everything just dies out, class two, it goes to a steady state, class three, it continues to produce sort of randomness forever. And then then there was class four which was sort of intermediate where it didn't just uh uh become constant, it didn't just produce kind of uniform randomness. Instead, it breaks into these localized structures that have complicated interactions.”
Electronic analog computers from the 1980s used passive LRC (inductor-resistor-capacitor) circuits to solve differential equations; differential equations from physical systems (springs, bouncing balls) could be mapped to circuit equations, allowing the physical circuit to simulate the system.
“in those days, so those analog computers were electronic and they had um uh like, you know, any old passive electronic circuit, uh you know, LRC circuit, inductor resistor capacitor circuit, there's a differential equation associated with that circuit. And you know, if you want to solve the equation for a ball bouncing on a surface or something like that, you can actually that's a little bit weird cuz it has a a weird condition to it, but let's say you want to solve, I don't know, a spring bouncing up and down. You want to solve that differential equation. You can convert that differential equation to an equation um to something where you just have characteristics in electric circuit, then you can just see what the time behavior of the electric circuit is and it will give you what those equations were like.”
Feynman was diagnosed with cancer multiple times; on the first diagnosis, Wolfram discussed with him the idea of understanding his medical condition from first principles, which Feynman was uncharacteristically not very interested in pursuing in his own medical case.
“he got uh, um, was got cancer several times and I remember talking to him the first time he had that um... I remember talking to him about in the case of the in that medical kind of case about sort of the first principles understanding of what's going on, which he actually wasn't so much into in that particular case, which was unusual for him”
Neural networks solved the previously hard problem of image recognition (e.g., distinguishing cats from dogs) by making it "really easy" through algorithmic and architectural advances, exemplifying how algorithmic breakthroughs enable new capabilities.
“neural nets where it was oh, we want to recognize there's a cat or a dog in this image. That used to be really hard, and now it's really easy”
Whether neural networks should be implemented in analog circuits depends on precision requirements; if neural networks require only 4-bit precision, analog circuits offer little advantage; if they require 32-bit floating-point precision, analog circuits might be beneficial.
“people certainly tried this is, you know, can neural nets be made analog?... it doesn't actually matter that there are all those digits of precision in the neural nets. So, it's kind of like, well, if you were worrying about making 32-bit floating point numbers, then yes, it might be worthwhile to have them in analog circuitry. If it turns out it's 4-bit numbers, it's less important to be in analog circuitry”
Richard Feynman was disparaging of philosophy, and his son Carl made a point of majoring in philosophy at MIT partly because the father had so vocally disparaged it—both father and son found this amusing.
“Feynman always uh, used to kind of disparage philosophy. Uh, one of the amusing sort of footnotes, his son Carl went to MIT and made a point of majoring in philosophy partly because uh, the old man had had so uh, uh, kind of um, uh, vocally disparaged philosophy. I everybody thought that was pretty amusing. I mean, at least the father and son seemed to both think it was pretty amusing.”