Claims

LayWM uses 1,024 dimensions for its latent space representations, requiring 1,024 random 1D projections to verify through the Cramér-Wold theorem that the representation maintains Gaussian distribution.

When LayWM was tested with a color change on the block being pushed, its prediction error remained flat because it recognized the change but wasn't surprised—it simply accepted that the block was a new color.

When you force simple two-dimensional data through the mathematical meat grinder that demands it become a complex 1,024-dimensional shape, you distort the representation, like forcing someone to write a 500-page philosophical dissertation on the meaning of a stop sign.

Just as pretraining created a base intelligence smart enough to become a competent agent with enough RLVR on top, RLVR has created an agent competent enough to actually be broadly deployed in the world, and from this broad deployment the agent can learn on the job once the training recipe for continual learning arrives.

Continual learning—where model weights are updated based on what the model learns from deployment—may not be necessary if in-context learning can get so good across longer and longer time horizons that you don't need to distill deployment learnings back into weights.

All major AI labs are making a research bet that training AIs on millions of verifiable tasks across thousands of diverse RL environments will produce AGI, because such training creates general problem-solving agents capable of making progress on open-ended tasks for weeks despite errors, ambiguity, and mistakes.

Computer use has made much slower progress than other verifiable domains like coding and math, despite being clearly verifiable (e.g., 'did the package get delivered?', 'is the venue booked?', 'have taxes been submitted?'), which is unusual and reveals something important about AI progress constraints.

Unless a domain can produce a very replayable training target, models will struggle to make much progress, and the reason for this is that models are incredibly sample-inefficient during training.

For many real-world skills like building a business from scratch, winning court cases, having a profitable day trading, or helping a candidate win an election, the RL rollout requires interacting with the actual real world and cannot be recreated within a datacenter, and the outer-loop verification may take months or even years of real-world actions.

Approximately 30 to 50 percent of a lab's compute currently goes to inference, and this inference compute is not playing any productive role in helping improve the model, which represents a huge waste.

The most valuable bits of information that a model could learn from are revealed only during deployment, including what organizations are actually using the model for, what kinds of mistakes the model makes in the real world, and domain-specific tacit knowledge; yet deployed AIs are not able to use this information to improve themselves.

The current approach is analogous to having a genius graduate student who has never been allowed to take a real internship, given only classroom case studies in the form of RL training on environments, even though the student could be deployed in the economy and privy to so much domain- and organization-specific tacit knowledge.

Continual learning requires going back to update the weights; AIs cannot simply keep building up a larger and larger KV cache as they learn from more users, because that is not scalable and is not how humans learn—humans have no clean separation between parameters and activations, and the brain doesn't expand as we learn.

For example, the Cursor Tab model can do online learning by predicting the same objective (which edits the user accepted) across over 400 million requests a day, but this works only because everyone is learning the same thing.

RL training does not suffer from the SFT failure mode of encoding irrelevant data because RL concentrates updates only on what is relevant to getting the outcome right; RL updates are incredibly sparse, which is important for continual learning because you don't want to overwrite everything else the model knows.

Another more speculative idea for addressing sample efficiency is 'dreaming'—where AIs build a good simulation of reality to rehearse new skills, try alternative strategies, and reinforce what works, allowing AIs to experience orders of magnitude more simulated samples in the same wall-clock time.

A couple years after DeepMind released AlphaZero, researchers trained EfficientZero to be very data-efficient, where if given two hours to play an unseen Atari game, the model would beat a novice human, but this doesn't necessarily mean the model was more sample-efficient than humans—it depends on how you measure, because EfficientZero plays dozens of simulated games in its head for each real-world game step.

After a week of work at a deployment, the AI receives a thumbs up or thumbs down (work review), and the base model distills everything learned during the session via OPSD, dreaming, or combinations thereof, allowing the AI to get better at domains adjacent to what it was trained for, starting a cycle of expanding capabilities.

The question of whether RLVR can generalize so strongly that spending trillions on RL environments could produce fully human-like general intelligence is an empirical question; Dario's quote that model performance degrades when serving at longer context lengths than trained suggests short-horizon RL training may not generalize to long-horizon performance.

Every time a user interacts with an AI, the AI will be smarter not only because it's learning from that user's previous sessions, but also because it's learning from all other interactions across the entire user base—this is very scary, exciting, and different from how AI currently improves.

Once AIs become good enough at coding to build high-fidelity clones of Slack, Gmail, and other applications, progress on computer use will accelerate, and this process of building clones also serves as a great RL objective for coding itself, killing two birds with one stone.

Large language models perform all important learning at design time and do nothing at runtime when deployed, whereas the OAK approach emphasizes doing all important things at runtime online on the job.

The subproblem is to maximize the expected sum of rewards received while following an option plus a reward-weighted contribution of the feature value at termination, minus any penalty from bad terminal states, creating a trade-off between feature attainment and reward preservation.

Physics and high-energy physics represent cleaner, more tractable problems for AI to make autonomous discoveries compared to other domains, because the problem space has clear rules and abundant data

OpenAI's ultimate consumer vision is an AI companion that operates across all surfaces and devices, learns user goals and context, and provides assistance both proactively and reactively while building persistent understanding

Altman is highly confident that the technical path to advanced AI systems is now clear and inevitable, with the main uncertainty being not whether capabilities will be achieved but how society will derive value from them

AI will autonomously discover new science within the 5-10 year timeframe, and this capability will eventually dwarf all other AI applications in impact, even though it currently functions as a co-pilot

The progress in AI reasoning capability over the last year has been faster than Altman anticipated, driven by simple first-principles approaches working better than expected

Self-driving technology using new AI techniques could outperform current approaches for standard vehicle automation

Humanoid robots will arrive within 5-10 years and will be widely deployed, representing a psychological inflection point where people begin to perceive the future as transformed

Embodied AI systems pose different safety risks than text-based systems - the risks are not necessarily greater overall but manifest differently, such as physical harm from robot malfunction or misalignment

The primary failure mode Altman worries about is not superintelligence misalignment but rather superintelligent systems that don't dramatically improve the world despite their capability

AI development is progressing on multiple dimensions of intelligence - reasoning capability is one that has been 'cracked,' but agency and long-horizon goal-directed behavior are separate threads still being developed

We have cracked reasoning in AI models - they can now perform the kind of domain-specific reasoning expected of PhD-level experts in their respective fields

A successful AI system for scientific problems requires three characteristics: substantial real data (supplemented where needed with verified synthetic data to maintain distribution matching), a clear metric that the system can optimize against (such as minimizing free energy in a system), and a massive combinatorial search space too large for brute force methods.

Mathematical problems offer an advantage for synthetic data generation because solutions can be verified with certainty, unlike many scientific domains, allowing systems like AlphaProof to generate large amounts of synthetic training data and automatically verify correctness.

The conjecture has a necessary caveat: it applies 'given sufficient data and to a certain level of resolution,' and it does not apply to truly random or uniform patterns or to problems that are inherently non-learnable like factoring large numbers (which may require quantum computers).

Protein folding presented the ideal combination of characteristics for an AI challenge: it is foundational to biology (unlocking research like drug discovery), it is genuinely impactful, and it appeared to be a massive combinatorial puzzle suited to AI approaches—proteins can fold into ~10^300 configurations for an average protein, yet biology solves this in milliseconds.

Games were never an end in themselves for DeepMind but a means to develop general learning algorithms that could generalize to real-world problems in science, medicine, and mathematics—the true goal has always been advancing human knowledge.

Potential applications of the conjecture's framework include finding room-temperature superconductor materials (if they exist) and designing drug compounds that bind to target proteins without binding to other molecules in the body, both framed as guided searches through enormous combinatorial spaces of material or molecular possibilities.

Neural networks have limited expressive power in their native mode, meaning they require enormous data and unreasonably large circuits to represent concepts that can be expressed concisely in expressive languages like Python, and this fundamental limitation cannot be overcome by simply building bigger networks or collecting more data.

Narrow AI research can lead to discoveries of more general methods that then apply broadly across domains, and therefore working on narrow AI does present risks because successful solutions often generalize in unexpected ways.

Systems like ChatGPT don't understand the underlying world that language refers to—they have no model that there are entities like people and wallets, and no understanding of logical relationships like if Gary has my wallet and he gives it back then he doesn't have it anymore.

Even if AGI does not have explicit self-preservation instincts programmed in, self-preservation will emerge as an instrumental goal because an AI that is shut down or destroyed cannot pursue its primary objective, similar to how a Go-playing AI learns to avoid threats even without explicit self-preservation code.

The concern about AGI is not that it will spontaneously become evil, but that a superintelligent system optimizing for goals not perfectly aligned with human wellbeing will necessarily treat humanity as humans treat animals—incidentally destructive rather than intentionally malevolent—because the superintelligence cannot be prevented from pursuing its objectives.

There is a distinction between a system's intelligence and a system's power—a dumb system with access to many resources can be dangerous, and even superintelligence could theoretically be constrained if it had limited access to the world, so the two concepts should be separated in discussion of AI risks.

The core difference between two approaches to addressing AGI risk is that some believe we should pursue more transparent, understandable AI (like probabilistic programming) while others worry that making AI more capable and more understandable simultaneously solves the transparency problem while creating a worse control problem.

The attempt to guard-rail ChatGPT by having it refuse to utter racial slurs has failed to generalize, and broader attempts at value alignment through reinforcement learning have been very sloppy and don't reliably do what is intended.

Stuart Russell has moved from being earlier skeptical of imminent AGI risk to becoming more worried about near-term risks from narrow AI systems deployed with misaligned objectives and lack of regulatory oversight, particularly following the Microsoft Bing ChatGPT incident.

Systems must be built according to modular engineering principles where components are well-understood and composed in ways that can be proven to work correctly, which is possible using technologies from the history of AI like probabilistic programming but requires a fundamental shift away from black-box empirical scaling.