Claims

The history of AI contains overlooked techniques and paradigms from 'paths not taken' (symbolic AI, logic programming, behavioral AI, agent-based systems) that were not fundamentally wrong but came too early or were abandoned due to paradigm shifts; these deserve rescuing for insights they might contribute to current and future AI research.

The second wave of AI (expert systems, 1980s) was based on the principle that intelligence is primarily a problem of knowledge; the key was to extract domain expertise from human experts and encode it as rules, then use those rules to make decisions (exemplified by the MYCIN system for diagnosing blood diseases).

Logic programming (exemplified by the WARPLAN system in Prolog) was theoretically elegant—the idea was to express all knowledge as logical statements and let automated deduction handle inference; WARPLAN solved complex planning problems in 15 lines of code versus thousands of lines required in imperative languages.

Rodney Brooks questioned the fundamental symbolic AI principles (knowledge and reasoning) and proposed instead that intelligence emerges from the interaction of multiple simple behaviors, many genetically hardwired through evolution; he emphasized embodied intelligence and reactivity over abstract reasoning.

Brooks built robots using a layered behavior architecture starting with obstacle avoidance, then adding exploration, then trash-finding; the approach worked in real robotics (exemplified by Roomba robots) but hit limits when trying to organize many behaviors and reason about their interactions.

Agent-based AI (early 1990s) reconceived software from a passive recipient of user commands (like Microsoft Word) into an active agent cooperating with and acting on behalf of the user; this is the conceptual ancestor of modern voice assistants like Siri and Alexa.

Multi-agent systems (Waldridge's main research focus) extend the agent concept: multiple AI agents could cooperate, negotiate, and delegate tasks to each other (e.g., my Siri could directly contact your Siri to arrange a meeting rather than me calling you).

The agent-based paradigm is agnostic between symbolic and connectionist (neural) approaches—agents can be implemented via either explicit rules or learned representations; the paradigm abstracts over the question of whether intelligence comes from modeling the mind or modeling the brain.

LLMs are not reasoning or solving problems from first principles; when you change the terminology in a problem statement to words the model has never seen in training, its performance collapses, suggesting it performs pattern matching on familiar phrasings rather than abstracting underlying problem structure.

The Transformer architecture was designed for next-word prediction and does not obviously generalize to logical reasoning, robotic AI, or embodied interaction; there is no reason to believe that increasing scale alone will overcome these architectural limitations.

The shift from exact/optimal algorithms to data-driven statistical approximation in AI is a 'sobering' or even 'depressing' lesson: the chief source of progress is not scientific insight but brute-force increases in compute and data (the 'Bitter Lesson' in AI research).

Despite the 'bitter lesson,' there is still 'magic' in AI worth pursuing: understanding the principles governing how LLMs work, their capabilities and limitations, and the fundamental laws underlying these systems is now practical experimental science rather than pure philosophy—a watershed moment.

Neural networks are a 'hack'—an engineering solution that works well at scale but is not based on deep cognitive science or philosophical theories of intelligence; they were inspired by brain structure but are not faithful recreations of how brains actually work.

It is unclear whether neural networks as currently designed are the right substrate for logical reasoning and abstract problem-solving; these capabilities might require different architectures or additional mechanisms not present in Transformers.

The 'microworld' problem in early AI: researchers built simplified simulated environments (e.g., simulated robot warehouses) where their systems worked well, but failed to transfer solutions to real-world problems because they had abstracted away all the difficult complexity.

Current AI research focus on mapping the capabilities and limitations of large language models—what they reliably can and cannot do—is one of the most important areas of science right now; models behave in 'weird ways' where small prompt changes yield dramatically different outputs, making this characterization difficult.

Waldridge works on multi-agent systems and is trying to understand how large language models can be deployed within agent architectures—specifically whether LLMs should handle only natural language or also problem-solving roles; this is at the cutting edge of AI research.

The history of computing suggests that the future of AI will necessarily involve multiple AI systems interacting with each other, not isolated systems; this follows from patterns in how computing has evolved toward networked, distributed architectures.

The singularity narrative—that machines will become superintelligent, recursively self-improve, and spiral beyond human control—is deeply implausible because it requires that we give AI autonomous control without safety guardrails, which would be irrational policy, and current AI systems show no evidence of developing independent goal formation.

The real risks from AI in the next one to two decades are not existential but tractable: AI-generated content will dominate the internet making it impossible to distinguish real from fake, societies will fragment around algorithmic echo chambers, and autocratic states will use AI-generated fake news to drown out truth.

Waldridge did not initially expect GPT-3 and current LLMs to be as capable as they turned out to be; AI researchers who claim they were unsurprised are being misleading—the phenomenological power of LLMs genuinely exceeded expert expectations.

The Transformer architecture (released by Google in 2017) was designed for token prediction—given a prompt, predict the next word—and represents a paradigm shift from rule-based to data-driven AI, from exact/optimal algorithms to statistical pattern matching on massive datasets.

Large language models like GPT-3 and GPT-4 are trained on 'all the digital content in the world' and can discuss any topic—quantum mechanics, history, recipes—with impressive fluency, but this does not mean they are doing reasoning; they excel at tasks with abundant training data where low-stakes errors are tolerable, not at real-world embodied tasks like robotics.

There is melancholy in discovering what AI techniques have and haven't worked: success came not through philosophy, cognitive science, and logic (the elegant symbolic approaches) but through mundane mathematics (statistics, probability, linear algebra) at scale.

The debate around whether large language models can actually do arithmetic shifted recently—GPT-4 can perform arithmetic at a higher success rate—but the fundamental question remains: is the model solving arithmetic from first principles or recognizing patterns of arithmetic from its training data?