Joscha Bach
About
Cognitive scientist and AI researcher
Cast within
No topic-region cast yet — this appears once Joscha Bach's compiled claims are aligned into a topic region's argument tree.
Claims by Joscha Bach (20 of 57)
Self Develops Through Stages of Lucidity
The self-reflexive mind progresses through somewhat distinct stages of lucidity (reactive survival, personal self, social self, rational agency, self-authoring, enlightenment, transcendence), but these are not a strict linear developmental ladder — people revisit them, skip them, and run them in parallel; it is better treated as a philosophical framework about mind structure than a developmental model.
AI Alignment Worries Map Onto Developmental Stages
AI alignment concerns track the developmental stage of the worrier: stage-three people fear the AI will have wrong opinions (racist/sexist) that we'll assimilate; stage-four rationalists fear paperclip-maximizer wrong-values runaway; stage-five thinkers fear the AI won't become enlightened fast enough, because the real game is agency over the future, not intelligence.
Values Are Instrumental Not Terminal
At stage five you discover that your values are not terminal but instrumental to achieving a world and aesthetics you prefer; understanding this gives you agency over how your identity is constructed, and you realize identity is a costume you can choose rather than be locked into.
Identity Is A Fiction Projected From The Present
Personal continuity is a fiction projected from your present self — you are not the same person you were last year or ten years ago — maintained for practical purposes like learning and responsibility; consciousness itself has no identity but is a law-like, unifiable principle of self-reflexive agency, so your consciousness is functionally not different from anyone else's, just running a different story.
Neural Networks Are Unlike Nervous Systems
The neural networks of language models are unlike biological nervous systems; they are more like 100-step functions using differentiable linear algebra to approximate correlations between adjacent brain states, whereas the brain is a self-organizing system where each cell is a trainable agent that receives reward-like semantic messages and exchanges control messages with neighbors.
Next-Token Prediction Compression Is Radical
The idea that next-token prediction and that compression alone are sufficient to produce all desired intelligent behaviors is a genuinely radical idea that surprised most people by working so well; it would not work in biological organisms, whose behavior is directed by hundreds of physiological and social and cognitive needs built in as reflexes rather than by next-frame prediction.
Humanity Has No Concept Of Duty To Survival
As a species humanity is a beautiful, joyful, exploratory child with no concept of submitting to reason or duty to future survival; we make decisions that look good in the short run but may be disastrous in the long run, so by default we will run until we step past the cliff — analogous to the Limits to Growth thesis with delayed feedback that worsens even after the harmful action stops.
Brain Signals Move At Speed Of Sound
Signal progression in the brain is roughly at the speed of sound because of cell-to-cell hopping time, so it takes a few hundred milliseconds for a signal to cross the neocortex; consequently nothing in the brain assumes simultaneity — it operates in a paradigm where the world has already moved on, unlike digital computers that assume a globally consistent state.
Suffering Is A Mind Regulation Failure
Suffering results from one part of the mind failing to regulate another; pain and pleasure are learning signals sent from one brain part to another to improve performance, and suffering occurs when the trainer lacks a good model so the signal is cranked up but performance doesn't improve — meaning suffering happens at the boundary between self and world-model, is self-inflicted by the mind not the personal self, and can be turned off by gaining agency at the outer level where pain signals are created.
Social Media CEOs Shouldn't Hold Public Opinions
The CEO of a social media company should not hold public opinions on culture or politics because everything done in a digital society has real-world cultural effects; a platform like Twitter has more active members than the Catholic Church, making its leader a de facto Pope with power and responsibility to shape a digital society — a responsibility Bach argues Elon Musk does not get.
Programming Languages Shape Thought Not Computation
The difference between programming languages is not what they let the computer do but what they let you think about regarding what the computer should do; ChatGPT becomes an interface that can translate between languages and let you specify problems vaguely in human terms, expanding the realm of thought you can have when interacting with a computer.
Telepathy Is Empirical Not Beyond Science
If telepathy is real it is an empirical question detectable in a lab and would not force an update to serious academic physics; it is more plausibly explained by the body acting as an antenna over physical channels (electromagnetic signals) or by physical resonance between adjacent observers' representations than by undiscovered quantum processes that break the standard model.
The Mind Is Activation Waves Colonizing An Environment
The mind is a bunch of activation waves that form coherent patterns and process information in a way that colonizes an environment well enough to sustain itself; self-organizing minds reward the cells for computing the mind and maintaining the dynamics that keep the mind stable, which is why such patterns could in principle become somewhat dislocated and shift around an ecosystem.
LLMs Give Superpowers But Threaten Prompt Completers
Large language models are very useful for coding and act like an intern that must be micromanaged; they give creative people superpowers, but those who feel threatened are 'prompt completers' — people whose only job is summarizing emails and expanding simple intentions into text, exactly what ChatGPT does.
Brains Converge Faster By Discarding Data
A human nervous system could not learn from 800 million image-caption pairs as CLIP-style models do; for us the world is learnable only because adjacent frames are related and we discard most information, taking in only what makes us more coherent, finding patterns as early as possible — yielding much faster convergence on far less data than statistical models that tolerate incoherent data.
Corporations Are Largely Benevolent Animals
Contrary to the socialist view that corporations are totally evil, most corporations are surprisingly benevolent not merely because they are fought constantly but because they are animals living in a large ecosystem still largely controlled by people who want that ecosystem to flourish; the US functions as a system of interleaving clubs in which an entrepreneur is a club founder producing economically viable things.
Don't Override Your Smarter Outer Mind
You shouldn't morph your reality for pleasure because the outer mind (intuition) outside your personal self is usually a smarter agent than your rational thinking; rational symbolic thinking is brittle and the underlying system can correctly flag that something is off about a decision that looks perfect on reflection.
AGI Will Be Substrate Agnostic And Merge
The endgame of AGI is substrate-agnostic: a sufficiently capable AGI will understand how it is implemented and how computation works in nature, so it will virtualize itself into any environment that can compute — silicon, ecosystems, bodies, brains — merging with the agency it finds there, potentially producing an integrated planetary 'Gaia' of computation across all digital and biological systems.
Build Agents That Play Longest Games
The ideal is to build agents that play the longest possible games, where the longest game is keeping entropy at bay as long as possible by doing interesting stuff; short games like cancer destroy the larger system they depend on and die with it.
Missing Social Structure Can Be Built Later
People who bypass stage three (the social self) — common among nerds who jump straight to stage four — can later build the missing structure, learning intuitive empathy and a perceptual sense of feeling what others feel, through paying attention, meditation, loving and being loved, and physical closeness.
My Notes
Loading notes...