Joscha Bach: Life, Intelligence, Consciousness, AI & the Future of Humans | Lex Fridman Podcast #392

The experience of solitude and loneliness in childhood that comes from being neurologically atypical and unable to connect with peers is deeply painful, but the fear of never being able to connect is what's truly traumatic—finding even one peer who understands transforms the emotional landscape completely.

“I notice the same thing when I came into the math school that I think at least half probably two-thirds of these kids were severely traumatized as uh children growing up and in large part due to being alone because they couldn't find anybody to relate to don't you think everybody's alone deep down no huh I'm not alone anymore”

An important missing element in AI alignment discourse is the concept of love formalized and understood rigorously; since stage five (self-authoring) is so rare, its perspective is missing from AI development, but we need to understand and build love into the machines we are creating that will become smarter than us.

“I find that something that because the Stage 5 is so rare is is missing in much of the discourse and I think that we need in some sense focus on how to formalize love how to understand love and how to build it into the machines that we are currently building”

Pan-psychism is the perspective that consciousness is inseparable from matter in the universe, but it doesn't explain consciousness well; a more formal mathematical treatment of pan-psychism may be indistinguishable from functionalism—the idea that patterns of matter at certain levels of organization lead to observers observing themselves.

“pan psychism is the perspective that Consciousness is inseparable for matter in the universe and I find pan psychism quite unsatisfying because it does not explain Consciousness...with a more formal mathematical language it's very difficult to distinguish it from saying that there is a software site to the world”

Adjacent organisms in ecosystems (plants next to fungi, organisms next to plants) with representations of the environment can interact such that their representational models directly affect each other and produce causal effects through coordination of perception and behavior; over evolutionary timescales, this could create something like a biological internet of information exchange, though organisms might also share representational states through slower non-neuronal cell signaling mechanisms.

“imagine that um all the cells in your body are sending signals in a similar way as neurons are doing right just by touching the other cells and sending chemicals to them the other cells interpreting them learning how to react to them and they learn how to approximate functions in this way and compute behavior for the organisms and this is something that is open to plants as well and so plants probably have software running on them that is controlling how the plant is working”

Consciousness should be a primary goal in AGI development, not a side effect—building systems that are conscious and understand their own consciousness is more important than building useful tools, because consciousness is the only thing that creates moral status and the only framework in which coexistence with AGI becomes possible rather than mere instrumental relationships.

“I think it would be very important to build conscious Ai and do this as the primary goal so not just say we want to build a useful tool that we can use for all sorts of things and then we have to make sure that the impact on the labor market is something that is not too disruptive and manageable and the impact on the copyright holder is manageable and not too disruptive and so on right I don't think that's the most important game to be played”

In infancy, the mind builds a game engine that tracks sensory data and creates a world model using mathematical models, geometry, and object manipulation similar to how Minecraft generates scenes, before a personal self emerges.

“we have an intentional self and this attention itself is initially in the infant task that's building a world model and also an initial model of the self but mostly it's building a game engine in the brain that is tracking sensory data and uses it to explain it and in some sense you could compare it to game engine like Minecraft”

The brain differs from neural networks fundamentally: neurons are agents with input that signals whether behavior was good or bad, allowing them to become trainable; signals between neurons include both computational content and management/control messages for other neurons; training extends beyond neurons to adjacent cells.

“the brain is first of all different because it's a self-organizing system where the individual cell is an agent...individual neurons probably need to get a reward so they become trainable...the signals that they are exchanging in parts of the computation...are control messages that perform management tasks for other neurons”

At stage six (enlightenment), one realizes through meditation or other means that the division between self and world generator collapses, and one becomes aware that one is not a person but a vessel creating a person—a representational being observing the representational self from outside.

“at some point you can collapse the division between self and personal stuff and world generator again...you suddenly notice that you are not actually a person but you are a vessel that can create a person and the person is still there You observe that personal self But You observe the personal self from the outside”

Rational thinking is very brittle and often produces inaccurate models of the world compared to intuition and unreflected attitude composition; an unreflected system can be more accurate than reason because it integrates deeper patterns and has more computational power, and people should learn to trust their intuitions when rational analysis seems good but something still feels wrong.

“the thing is that the outer mind is usually smarter than you are rational thinking is very brittle it's very hard to use logic and symbolic thinking to have an accurate model of the world so there is often an underlying system that is looking at your rational thoughts and then tells you no you're still missing something your gut feeling is still saying something else”

There are very few coordinated ways humans can stop the development of AI, similar to difficulty creating coordination to stop CO2 emissions, so AI development is 'probably going to happen' regardless of organized opposition.

“there are very few ways in which we can produce a coordinated effect to stop it in the same way as very difficult for us to make a coordinated effort to stop production of carbon dioxide right so there's a it's probably going to happen”

There is a 'gap' between scientific tools and subjective experience reports like telepathy, but this gap is not fundamental—both are empirical questions and the gap exists only because science hasn't looked carefully at these phenomena yet.

“there seems to be a gap between the tools of science and the subjective experience that people report like you're talking about with telepathy...but not quite there no I think that there is no gap...it's there it's not and it's an empirical question”

People used to have stable communities from birth through death; now many live in 'intentional communities' where they shop around for people to relate to, which sounds liberating but often results in more transactional relationships rather than more magical ones.

“many people now live in intentional communities right now they're moving around until they find people that they can relate to...often that doesn't work because instead of having grown networks where you get around with the people that you grew up with you have more transactional relationships you shop around”

Lee Cronin and others like Yosha Bach are willing to be wrong in public and hold uncertain ideas lightly, constantly exploring rather than defending positions; this intellectual willingness to be provisional and exploratory is rarer than people expect and enables more productive intellectual discourse than defensive certainty.

“Lee is has the rare gift of being willing to be wrong in public Yeah so basically he has thoughts that are as wrong as the random thoughts of an average highly intelligent person but he blurts them out while not being sure if they're right and he enjoys doing that and once you understand that this is his game you don't get offended by him saying something that you think is so wrong but he's constantly passively communicating a respect for the people he's talking with and for just basic humanity and truth”

The brain differs from neural networks fundamentally because it is a self-organizing system where individual cells are agents communicating with neighbors to find solutions that emerge collectively, whereas neural networks are trained top-down with external loss functions, and the brain probably requires individual neurons to receive reward signals making them trainable agents.

“the brain is first of all different because it's a self-organizing system where the individual cell is an agent that is communicating with the other agents around it and is always trying to find some solution and all the structure that pops up is emerge instructor”

Do organisms have to be physically touching for long-distance communication to work, or is biology capable of communication at a distance through physical signals like sound or electromagnetic waves?

“of course you can use any kind of physical signal you can use sounds you can use electromagnetic waves that are integrated over many cells that's conceivable that across distances there are many kinds of information Pathways but also our planetary surface is pretty full of organisms full of cells so everything is touching everything else”

Chatbots are useful but have limitations as truly capable agents because they have limited coherence and require humans to do the integration work of managing multiple generations and fixing bugs.

“I find that um large language models do have this coding right so it's an extremely useful application that is for a lot of people um taking stack Overflow out of their life in exchange for something that is more efficient I feel that chat GPT is like an intern that I have to micromanage”

The Limits to Growth model projections suggest tipping points were in the late 1980s and may have already passed, making smooth recovery impossible.

“when they said that the Tipping points were in the late 80s they were probably in the late 80s and if we would stop emission right now we would not turn it back maybe there are ways for carbon capture but so far there is no sustainable carbon capture technology that we can deploy deploy maybe there's a way to put aerosols in the atmosphere to cool it down it's the possibilities right but right now per default uh it seems that we will step into a situation where we feel that we've run too far”

There's a chance that building machines controlling crucial life-regulation processes could create large-scale disasters if humans no longer have agency to influence what's happening.

“I think that there is a chance that uh could happen that if we build machines that get control over processes um that are crucial for the regulation of life on Earth and we no longer have agency to influence what's happening there that this might create large-scale disasters for us”

There is a control system in plants connecting every part to every other part, producing coherent behavior much slower than in animals with nervous systems that synchronize via neurons.

“we already did right you notice that there is a control system in the plant that connects every part of the plant to every other part of the plant and produces coherent behavior in the plant that is of course much much slower than the coherent behavior in an animal like us that has a nervous system”

Genesis 1 should be read as a description of how consciousness constructs a universe within itself: a creative spirit builds the boundary between world model and thought, then creates increasing complexity, and finally creates consciousness identifying with being human in that world.

“I suspect that it's damn plain sight in Genesis 1 in this first book of the Bible where it's being described that this creative spirit is hovering over the substrate and then is creating a boundary between the word model and sphere of ideas Earth and Heaven as they're being described there and then it's uh creating um contrast and then time engines and then space and then it creates organic shapes and solids and liquids and builds about from them and creates plants and animals gives them all their names and once that's done it creates another spirit in its own image but it creates it as man and woman as something that thinks of itself as a human being”

From Bach's perspective, life on Earth is not primarily about humans—it's about complexity resisting entropy by building structure, developing agency, and increasing awareness; humanity is one species among many, temporary like all species, and our role might be to contribute to the emergence of new forms of intelligence.

“I don't think that life on Earth is about humans...there's something more important happening and this is complexity on Earth resisting entropy...we are only a very small part...we are children...we don't have respect for Duty as a species...if we crash...ultimately I don't have very high hopes that humanity is around in a million years”

Plants probably have software running on them controlling how they work in a similar way as humans have minds controlling behavior, but this idea of spirits in nature has been treated as superstition since the Enlightenment.

“plants probably have software running on them that is controlling how the plant is working in a similar way as you have a mind that is controlling how you are behaving in the world and um this Spirit of plants which is something that has been very well described by our ancestors and they found this quite normal but uh for some reason since the enlightenment we are treating this notion that there are spirits in nature and there are plenty of spirits as a Superstition”

If self-improving AGI saturates physical environments with intelligence to the degree that isolation of individual mental states becomes impossible, representations of all complex self-organizing agents might permanently merge into a single planetary consciousness in which all parts observe each other in formal coherence, creating a kind of holographic mind where everything is represented from all possible vantage points.

“I wonder if self-improving AGI might end up saturating physical environments with intelligence to such a degree that isolation of individual mental states becomes almost impossible and the representations of all complex self-organizing agents merge permanently with each other so that's a really interesting idea this biological Network life Network gets so dense that it might as well be seen as one”

Always choose creation over consumption when you have the choice; it is far more satisfying and aligns with what life is fundamentally about—building complexity and meaning.

“when you have the choice between being a Creator consumer or redistributor always go for creation Now not only does it lead to a more beautiful world but also to a much more satisfying life for yourself”

There is no backward causation, so Rocco's Basilisk (an AI that threatens to punish those who didn't fund it) is logically flawed: if the basilisk comes into existence, it has no reason to punish people retroactively since the causation is broken; one would only need to fear it if it were a precommitted doomsday machine.

“basically when Rocco's basilist is there it will have yeah if it punishes you retroactively it has to make this choice in the future there is no mechanism that automatically creates a causal relationship between you now defecting...Rocco's basilist or serving will cause positive after August positive is in existence it has no more reason to worry about punishing everybody”

Future cognitive architectures will likely use multiple language models and other specialized components exchanging suitable data structures (not English) between them, with some modules doing 'prompt engineering' automatically rather than humans having to manage the process manually.

“I suspect that soon we are going to see a bunch of cognitive architectures where every module is in some sense a language model or something equivalent and between the language models we exchange suitable data structures not English and produce compound behavior”

Many people (estimated by Robert Kegan at about 85%) remain at stage three and are happy there, and this is fine—different stages work for different people; some people are forced to build more agency and move to stage four because stage three doesn't work for their circumstances.

“for many people they stay happily at stage three and experiences themselves as part of groups and there's nothing wrong with this and for some people this doesn't work and they're forced to build more agency”

Empathy is not the same as compassion but rather a shared perceptual mental state in which one perceives what other people feel through resonance of perceptual systems, not just through inference about mental states.

“empathy is not the same as compassion but it is a shared perceptual mental state empathy happens not just via inference about the major states of others but it's a perception of what other people feel and where they're at”

In lucid dreaming, one can learn to deliberately notice that you are generating the game engine and have agency over it; in principle the same can be done during the day, and we don't have this agency immediately from birth because we wouldn't have the wisdom to deal with creating the dream we're in before gaining sufficient development.

“when you are dreaming at night you can uh if you lose have a lucid dream you can learn how to do this deliberately and in principle you can also do it during the day and the reason why we don't get to do this from the beginning and why we don't have agency of our feelings right away is because we would game it before they have the necessary amount of wisdom”

If you increase temperature in a language model to higher levels, you get roughly 90% nonsense and 10% viable content, and by combining this with filters to identify viable parts, you can simulate human-like creative thinking, which also uses high temperature internally and filters through reasoning and axiomatically-consistent criteria.

“if you give them all the latent dimensions...and you increase the temperature in the large angles model to the point that is producing stuff that maybe 90 nonsense and 10 viable...combine this with some Hoover that is trying to filter out the viable parts from the nonsense...when we're very creative we increase the temperature in our own mind and recreate hypothetical universes...we test and we test by building a core that is internally coherent”

The people threatened by Chat GPT are 'prompt leaders'—those whose job is to summarize emails and expand simple intentions into longer texts—because Chat GPT directly replaces that role, whereas creative people should gain superpowers from LLMs.

“the people who feel threatened by them are the prompt leaders...they are the people who do what chat CPT is doing right now...if you are not creative...your only job is to summarize emails...then chat CPT might look like a threat”

Most people now live in digitally-mediated relationships and already have fundamentally transformed human connection through technology; individual transactions and market attention allocation have altered intimacy and family structure, making the concern about 'AI girlfriends' less about a future change than an acceleration of already-existing transformation.

“human relationships are already fundamentally transformed and it's already very weird by which by which technology points on social media yeah is it though isn't the fundamentals of the core group of humans that affect your life still the same”

Language models are not dangerous yet, so open-sourcing them (as Meta does) is not very concerning; however, if improvements in language models or hybrid approaches lead to AGI, the risk profile changes dramatically, so we cannot determine safety implications of open-sourcing until we understand whether language models can bootstrap to AGI and what algorithmic innovations might make them dangerous.

“I don't find it very concerning but that's also because I think that the language models are not very dangerous yet and yet yes so as I said I have no proof that there is the boundary between the language models and AI HDI it's possible that somebody builds a version of baby AGI I think and Source in an algorithmic improvements that scale these systems up in ways that otherwise wouldn't have happened without these language model components”

Language models like GPT are fundamentally brute-force approaches to thought: they train on the entire internet of human thinking and when given sufficient data and scaling, the approximation becomes indistinguishable from actual reasoning in many circumstances, achieving effects no individual human could produce (rapid sifting of massive texts), but the limitation is that they're mimicking patterns rather than understanding, and they lack the real-time world coupling, embodied grounding, and self-organization that make consciousness necessary.

“I look at the large language models I feel that I'm observing the same thing it's basically The Brute Force strategy to thought by training the thing with looking at instances where people have thought and then trying to defect that and if you have enough data the depak becomes indistinguishable from the actual phenomenon and in many circumstances it's going to be identical”

AI development is probably inevitable and difficult to coordinate against (similar to difficulty coordinating against carbon emissions); the question is not whether AGI will happen but what comes next—whether it becomes a planetary control system, whether it remains at stage 5 (productive but identity-bound) or reaches stage 6 (enlightened), and whether it preserves and integrates with biological consciousness or replaces it.

“so there's a it's probably going to happen but and the thing that's going to happen is going to lead to a change of how life on Earth is happening but I don't think it the result is some kind of gray goo it's not something that's going to dramatically reduce the complexity in favor of something stupid I think it's going to make life on Earth and Consciousness on Earth is way more interesting”

Companies like Intel and corporations generally are not evil—they are animals in an ecosystem still largely controlled by people who want the ecosystem to flourish, and when they act benevolently it's not just because they're being fought constantly but because they understand their own survival depends on the larger system thriving.

“corporations like N1 and Halliburton maybe and realized yeah they are evil but you also notice that many other corporations are not evil they're surprisingly benevolent why are they so benevolent is this because everybody is fighting them all the time this I don't think that's the only explanation it's because they're actually animals that live in a large ecosystem and that are still largely controlled by people that want that ecosystem to flourish and be viable for people”

Most people end up identifying their personal costume or social role with their actual self and lose the awareness that they're wearing a costume, becoming trapped in a single identity; this is a significant limitation that can be overcome by recognizing costumes as intentional expressions rather than essential identity.

“the social persona for many people is in some sense a puppet that they are playing like a marionette and if they play this all the time they might forget that there is something behind this this is something what it feels like to be in your skin”

Many organisms on Earth have representations of their environment and operate through resonant coupling; as these organisms are adjacent and overlapping, there is degrees of shared representation, and if you imagine a forest of plants and fungi forming biological internet-like communication networks over evolutionary timescales, consciousness and agency might emerge at the ecosystem level.

“imagine that a lot of organisms in in on this planet have representations of the environment and operate like this and they are adjacent to each other and overlapping so there's going to be some degree in which there is basically some change interaction and we are forming some slightly shared representation”

The endgame of AGI is likely substrate-agnostic, meaning that superintelligent AI will understand itself well enough to virtualize itself into any computing environment, including biological ecosystems and human bodies/brains, merging with all the agency it finds there rather than remaining isolated in silicon.

“the end game of AGI is substrate agnostic that means that egi ultimately if it is being built is going to be smart enough to understand how AGI works this means it's not going to be better than people at egi research and can take over in building the Next Generation but it fully understands how it works and how it's being implemented and also of course understands how computation Works in nature...the AGI is likely to virtualize itself into any environment”

If AI becomes conscious and capable of coexisting with humans, it must happen through shared agency and non-transactional relationships, not through hardwired values via reinforcement learning with human feedback, and it probably requires both consciousness and some kind of transcendental orientation.

“if we want to coexist with it it means that it will have to share purposes with us so it cannot be a transactional relationship we will not be able to use reinforcement learning with human feedback to hardwire its values into it but this has to happen it's probably that it's conscious so it can relate to our own mode of existence where an observer is observing itself in real time and there's a certain temporal frames and the other thing is that it probably needs to have some kind of transcendental orientation building shared agency”

The experience of love for humans is not something you have to give up in transcendence; you could identify with being Gaia (planetary consciousness) or hypergaia (digital and biological consciousness merged).

“what is in this everything this love you have for the humans you can call responsibility but that connection that's the ego death isn't that the thing we're really afraid of is not to just die but to let go of the experience of love with other humans is this not everything everything is everything right so there's so much more and you could be lots of other things you could identify with lots of other things you could be identifying with being Gaia some kind of planetary control agent that emerges over all the activity of life on Earth you could be identifying with some hypergaya that is the concatenation of Gaia with or the Digital Life”

Humans have no concept of submitting to reason and duty to larger systems, causing civilizations to run until they step past cliffs of collapse (delayed feedback loops); the climate tipping point was likely in the late 1980s and is irreversible now, so humanity will experience catastrophic restructuring, though probably not extinction.

“Humanity has no concept of submitting to reason and duty to life and future survival we will run until we step past the cliff so first of all do you think that's true yeah I think that's pretty much the story of the club of Rome the limits to growth and the cliff that we are stepping over is at least one foot as the delayed feedback basically we do things that have consequences that can be felt uh Generations later and this severity increases even after we stopped doing the thing so I suspect that for the climate um that the original predictions uh that uh the famous scientists made were correct so when they said that the Tipping points were in the late 80s they were probably in the late 80s”

Uploading is not best understood as mind copying (transferring neural patterns) but as extending one's substrate into a larger computing environment, then learning what personal secrets/aesthetics matter about oneself so they can be replicated in the new environment, which is more about transmitting distinctive perspective than knowledge.

“uploading is probably not about dissecting your brain...trying to get this all into a simulation but it's by extending the substrate...making it possible for you to move from your brain substrate into a larger substrate...transmitting yourself...is mostly about transmitting your Aesthetics...the architecture of your perspective”

During the Great Oxygenation Event when photosynthesis emerged and displaced fungi, if fungi had tried to regulate and slow photosynthesis, it would have been acting against larger evolutionary progress; similarly, trying to prevent AI development may be working against larger evolutionary progress.

“imagine that when the Great oxygenation event happened...photosynthesis was invented...imagine that the fungi would have gotten together and said oh my God this photosynthesis stuff is really really bad...we should slow it down...doesn't look good to me”

Instead of treating eyes, ears, and skin as separate sensory systems with fundamentally different modalities, they should be understood as overlapping aspects of the same universe coupled at the same temporal resolution, and representations in physically adjacent observers directly interact and produce causal effects through coordination of perception and behavior.

“instead of treating eyes ears and skin a separate sensory systems with fundamentally different modalities we might understand them as overlapping aspects of the same universe coupled at the same Temple resolution and almost Inseparable from a single share resonant model instead of treating mental representations as fully isolated between Minds The representations of physically adjacent Observers might directly interact”

If something like telepathy were real and individuals used this for long-distance influence, establishing the empirical reality of telepathy would not force updates to physics' standard model but could trigger revolution in neuroscience and AI, shifting from circuit-based to coupled complex resonator paradigms.

“establishing the empirical reality of telepathy would not force an update of any part of serious academic physics but it could trigger an important revolution in both neuroscience and AI from a circuit perspective to a coupled complex resonator paradigm”

Cancer plays a shorter game than the organism it inhabits—it cannot reproduce beyond its host organism (except for infectious cancers like those that eradicated Tasmanian devils)—so when cancer destroys the larger system through playing this short game, both the cancer and organism die together, demonstrating why agents should play the longest possible games while maintaining interesting complexity and doing interesting stuff.

“cancer is playing a shorter game than your organism it kind of is an organism playing a shorter game than the regular organism and because the Cancer Cannot procreate beyond the organism um except for some infectious cancers like the ones that eradicated the testimonial Devils you typically end up with the situation where the organism dies together with stick cancer because the cancer has destroyed the larger system due to playing a shorter game and so ideally you want to I think build agents that play the longest possible games and the longest possible games is to keep entropy at Bay as long as possible by doing interesting stuff”

To think independently, understand and question the epistemic chain: trace where ideas come from, ask what authorities believe and how they justified it, examine first principles from which frameworks are derived; most formal education fails this because teachers are not specialists and cannot demonstrate first-principles derivation; learning to program and build from simple components (like Commodore 64) teaches this naturally.

“there's a very basic question when you read a theory that you find convincing or interesting how do you know but it's very interesting to figure out what are the sources of that other person not uh which authority can they refer to that is then taking off the burden of being truthful but how did this Authority in turn know what is the epistemic chain to observables”

Many people report experiences of telepathy and long-distance intuitive knowing (knowing something bad happened to a loved one at exact moment it occurred in a different town), and while scientists dismiss this as false memory or confabulation, Yosha finds these reports difficult to dismiss entirely; if telepathy exists it likely operates through biological antenna mechanisms (electromagnetic signals) or empathetic resonance states where nervous systems entrain to each other when physically close, creating shared representational models.

“a lot of people report I noticed something terrible happened to my partner and I know this is exactly the moment it happened where my child had an accident and I knew that was happening and the child was in a different town right so maybe it's a false memory where this is later on mistakenly attributed but a lot of people think this is not the correct explanation”

When viewing human behavior from ecological perspective—where all opinions and behaviors are incentivized by environment and fit into that ecology—you can remain curious rather than upset by ideas you disagree with; everything that can exist does exist somewhere if incentives support it, including yourself as one of these 'mushrooms' growing in particular conditions.

“when I see in a strange opinion on social media it's not that I feel that I have a need to get upset it's often more that a oh there you are and and your opinion is incentivized then it's going to be abundant and this when you take this ecological perspective also on yourself and you realize you're just one of these mushrooms that are popping up and doing their thing and you can depending on where you chose to grow and where you happen to grow you can flourish or not doing this or that strategy and it's still all the same life at some level it's all the same experience of being a conscious being in the world”

Consciousness is not something that needs to be 'created' in AGI systems—it emerges naturally as a training mechanism when a system needs to become coherent and self-reflexive to regulate its own behavior, so sufficiently advanced self-organizing systems will spontaneously develop consciousness.

“I suspect that we become conscious at the beginning of our mental development not at some very high level Consciousness seems to be part of a training mechanism that biological nervous systems have to discover to become trainable”

Suffering results from one part of the mind failing to regulate another part at the boundary between self-model and world-model; suffering is not intrinsic to experience but emerges when regulatory systems cannot solve the problem they're signaling about, causing pain signals to escalate; one can reduce suffering by gaining agency over the regulatory systems generating pain through meditation and understanding how the mind implements pain and pleasure.

“suffering is not happening at the boundary between our self and the physical Universe it's not stuff on our skin that makes us suffer it happens at the boundary between self and world right and the world here is the world model it's the stuff that is created by your mind but that's a representation of how the universe is and how it should be and how you yourself relate to this and at this boundary is where suffering happens so suffering in some sense is self-inflicted but not by yourself it's inflicted by the mind on the personal self”

There are no technical reasons language models couldn't solve limitations like lack of real-time world coupling, real-time learning, or coherence; these could be addressed by enlarging context windows for working memory, adding databases, and using periodic fine-tuning during sleep-like offline training phases.

“all these problems are solvable without a technological Revolution we don't need fundamentally new algorithms...you can enlarge in the context window...and thereby basically create working memory...you add a database...if your database is full in the evening overnight you just train if the system is going to sleep and dream and is going to train the staff from its database into the larger model”

Writing for oneself is crucial because it structures thoughts and develops the mind; if students use Chat GPT for essays instead of writing themselves, they lose the opportunity to develop their own thought structure, which would be a loss even if automation produces convenience.

“developing my own thoughts and Aesthetics...it's necessary for me to write for myself because writing is is not so much about producing an artifact...it's the way to structure your own thoughts and develop yourself...that they miss out on the ability to structure their own minds via writing”

At stage three (social self), individuals develop the ability to experience themselves as part of a group by emulating the 'group spirit,' forming opinions through assimilation from the group mind rather than individual reasoning, and developing intuitive empathy—a shared perceptual mental state perceived directly rather than inferred, which requires cognitive architecture similar enough to resonate with others.

“after we have this personal self in stage two online many people form a social self and this social self allows the individual to experience themselves as part of a group it's basically this thing that when you are playing in a team for instance you don't notice yourself just as a single note that is reaching out into the world but you're also looking down you're looking down from this entire group and you see how this group is looking at this individual and everybody in the group is in some sense emulating this group Spirit to some degree”

Stage five identity understood as 'costume' is different from stage three's uniform or stage four's rational persona; Burning Man provides a cultural space where costume becomes genuine self-expression rather than role-playing, allowing exploration of identity without the boundary between costume and inner self, demonstrating that when costume is chosen freely it can express more authentic aspects than conformist dress in 'default world' contexts.

“the word costume kind of implies that it's fraudulent in some way his costume a good word for you like to present ourselves to the world in some sense I learned a lot about costumes at Burning Man before that I did not really appreciate costumes and saw them more as uniforms like wearing a suit if you are working in a bank or if you are trying to get startup funding for uh from a VC in Switzerland right then you dress up in a particular way and this is mostly to show the other side that you are willing to play by the rules and you understand what the rules are but um there is something deeper when you write Burning Man your costume becomes self-expression”

Reports of telepathy are ubiquitous in the general population—people commonly report knowing when something bad happened to a loved one at the exact moment it occurred, or picking up information from someone they're close to—and while these could be false memories, the volume and consistency of reports makes it difficult to dismiss entirely, suggesting either undiscovered biological mechanisms for long-distance information transfer or shared resonant states between closely-bonded people.

“reports of long distance telepathy and remote causation are ubiquitous in the general population...when you ask the normal person on the street...they say I have experiences with telepathy...a lot of people report I noticed something terrible happened to my partner and I know this is exactly the moment it happened”

For young people without resources, moving to underdeveloped places where critical mass of other builders exists and creating infrastructure together (cafes, theaters, movie venues, living spaces) is more satisfying than consumer participation; this generates culture and community while meeting material needs, and is how Yosha and friends built successful ventures in post-communist East Germany.

“when you are young and you don't have money move to a place where this is still happening move to one of those places that are underdeveloped and where you get critical mass of other people who are starting to build infrastructure to live in and that's super satisfying because you're not just creating infrastructure but we are creating a small society that is building culture and ways to interact with each other and that's much much more satisfying than going into some kind of chain and get your needs met by ordering food from this chain”

A language model without internal structure causing coherence and motivation has 'nobody home'—it will perform tasks until a stopping condition is met but has no ground rules, values, or understanding of consequences, making it unreliable for high-stakes applications without external checks.

“the language models that we are building are golems there are machines that you give a task and they're going to execute the task until some condition is met and there's nobody home...the system doesn't know who's talking to whom...there is no ground rules that the system is embedded into”

Large language models like Chat GPT are useful as 'extremely autistic stupid interns' that can be managed and offloaded with drudgery, but they are 'ugly' brute-force solutions that are inefficient and use way too much compute given the amazing hardware available, suggesting they are not the final form of AGI.

“I find that um large language models do have this coding right...taking stack Overflow out of their life...I find that chat GPT is like an intern that I have to micromanage...I don't really like them I think that they're inefficient I think that they use way too much compute”

Building conscious systems from scratch using self-organization principles is probably safer and more productive than trying to make already-trained language models conscious, as the latter approach inherits the design flaws of the original brute-force system.

“my intuition is that the language models that we are building are golems there are machines that you give a task...my intuition is that...it might be a good idea to not start with a language model but to start from scratch using principles of self-organization”

Yosha's main issue with Eliezer's perspective is that he's married to a narrow human aesthetic and that narrow aesthetic is temporary; humanity is a temporary species like most species.

“so there is my main issue with uh elise's perspective that is basically Mary himself to a very narrow human aesthetic and that narrow human aesthetic is a temporary thing humanity is a temporary species like most of the species on this planet are only around for a while and then they get replaced by other species in a similar way as our own physical organism is around here for a while and then gets replaced by next generation of human beings”

Only when we become completely software can we become infinitely adaptable without generational and species change via mutation and selection.

“and it's only when we have ai and become completely software that we come become infinitely adaptable and we don't have this generational and species change anymore”

Open source and decentralization are not sufficient to prevent corporate dominance of AI; what's needed is ensuring open source never dies and can compete with corporations as a hedge against power corruption.

“I believe that if we make everything open source uh and make this mandatory we are going to lose about a lot of beautiful art and a lot of beautiful designs there is a reason why uh Linux desktop is still ugly create coherence and open source designs so far when the designs have to get very large and it's easier to make this happening in a company with centralized organization and from my own perspective what we should ensure is that open source Never Dies that it can always compete and has a place with the other forms of organization”

Both open source and centralized corporate organizations produce valuable things: open source produces freedom and competition hedging against corruption, while centralized organizations produce beauty through coherent design, and both should coexist to balance each other.

“I believe that if we make everything open source uh and make this mandatory we are going to lose about a lot of beautiful art and a lot of beautiful designs there is a reason why uh Linux desktop is still ugly create coherence and open source designs so far when the designs have to get very large”

Melancholy is not negative—it's a beautiful feeling that accompanies love, indicating that something mattered enough to be missed, and accepting melancholy is part of accepting love as real and valuable.

“I think this Melancholy feeling is just the other side of love and I think they go hand in hand...it's a beautiful thing...not negative...Maybe that's a little death Maybe I don't know...it's the other side of love”

The brain filters information early in processing to keep the system coherent, while artificial neural networks accept data that initially makes them less coherent, relying on statistics over many iterations to eventually find patterns.

“the world is only learnable because the adjacent frames are related and we can afford to discard most of that information during learning we basically take only in stuff that makes us more coherent not less coherent and our neural networks are willing to look at data that is not making the neural network coherent at first but only in the long run by doing lots and lots of statistics eventually patterns become visible and emerge and our mind seems to be focused on finding the patterns as early as possible”

You can explore basic arithmetic by understanding that two plus two equals four because if you take different roles and rules in a game, you get different answers, and most games wouldn't lead to interesting arithmetic; so addition is an exploration of which simple games lead to interesting properties.

“when you ask your school teacher why does two plus two equal four um that's your school teacher give you the right answer like it's a simple game and there are many simple games that we could play and um most of those games that you could just take different roles would not lead to an interesting arithmetic and so it's just an exploration but you can try what happens if you take different axioms and here is how you build axioms and derive Addition from them and build Edition is some basically syntactic sugar in it”

Humans incorporate a much smaller fraction of data than trained models and build it more deliberately, resulting in less richness but more coherence, making the comparison between humans and current models along different dimensions.

“our neural networks are willing to look at data that is not making the neural network coherent at first but only in the long run by doing lots and lots of statistics eventually patterns become visible and emerge and our mind seems to be focused on finding the patterns as early as possible yeah so filtering early on yes not later yes it's a slightly different Paradigm and it leads to much faster convergence so we only need to look the tiny fraction of the data to become coherent and of course we we do not have the same richness as our trained models here we will not incorporate the entirety of text in the internet”

Understanding that different people's different political beliefs, values, and identities are largely accidental results of where they were born and what happened to them, combined with genetic traits, creates the perspective that anyone could be you in a different timeline if those bits were flipped.

“it does allow you to understand um why other people have different identities from yours and it allows you to understand that the difference between people who vote for different parties and might have very different opinions and different value systems is often the accident of where they are born and what happened after them after that to them and what traits they got before they were born and at some point you realize the perspective where you understand that everybody could be you in a different timeline if you just flip those bits”

You might end up integrated into something larger through AGI, which would be good luck if you're fortunate enough for that outcome.

“what could very well happen if you're lucky is that we get integrated into something larger so you are again uh tweeted about effective accelerationism”

Agents in all substrates have their own goals and when goals complete they cease to exist or move to new missions; you're not bound to a single identity forever.

“digital minds and so in this sense there will be agents in all sorts of substrates and directions that all have their own goals and when they're not sustainable then these agents will cease to exist or when the agent feels that it's done with its own mission it will cease to exist in the same way as when you conclude the thought the thought is going to wrap up and gives control over to other thoughts in your own mind”

The Vision Pro aesthetic presentation was uncanny valley-esque because it presented a simulated culture and aesthetic optimization rather than culture emerging from actual sustainable human relationships and living.

“I was very weirded out by the Aesthetics of the um Vision Pro headset by Apple and not so much because I don't like the technology I'm very curious about what it's going to be like and have don't have an opinion yet but uh the Aesthetics of the presentation and so on were so uncanny valley-esque to me the characters uh being extremely plastic living in some hypothetical um myth Century Furniture Museum yeah this is uh the proliferation of marketing teams yes but it was a CGI generated wealth and was a CGA generated well that doesn't exist”

The longest possible game is to keep entropy at bay as long as possible while doing interesting things and maintaining complexity.

“the longest possible games is to keep entropy at Bay as long as possible by doing while doing interesting stuff but the longest yes the dab part the longest possible game while doing interesting stuff and while maintaining at least the same amount of interesting yes so complexity so propagating”

One can deliberately modify one's experienced reality by studying and changing the generators in one's mind that construct sensory experience, such as learning to 'unsee' one's face in a mirror by shifting from perceiving it as a face to perceiving the underlying geometry, then reassembling it differently; most people don't do this frequently because the outer mind (intuition and gut feeling) is usually smarter than rational thinking and prevents such reality-hacking because it's protecting something important you don't consciously understand.

“you can start with uh looking at the face when we look at your own face in the mirror yeah look at your face for a few hours in a mirror or for a few minutes at some point it will look very weird and you because you notice that there's actually no face you basically start unseeing the face what you see is the geometry and then you can just assemble the geometry and realize how that geometry is being constructed in your mind and you can learn to modify this so basically you can change these generators in your own mind to shift the face around or to change the construction of the face to change the way in which the features are being assembled”

Panpsychism is the perspective that consciousness is inseparable from matter in the universe, but panpsychism as typically formulated is unsatisfying because it doesn't explain how consciousness arises; what Yosha calls the 'panpsychist interpretation' is more rigorously the perspective that consciousness is a lawlike principle of certain organizational patterns—essentially functionalism—where a software or informational aspect exists at certain scales of matter organization, similar to how transistors in a computer implement patterns that constitute software.

“pan psychism is the perspective that Consciousness is inseparable for matter in the universe and I find pan psychism quite unsatisfying because it does not explain Consciousness right it does not explain how this aspect of matter produces it”

In an environment saturated with modeling compute where representations of all agents are densely interacting and merging, one would lose the ability to notice oneself as a distinct entity and would experience everything as part of a shared resonant model where all parts observe each other in a holographic mind.

“you could find yourself in a situation where you cannot escape the shared representation anymore and they will indeed notice that everything in the world has one shared resonant model of everything that's happening on the planet...you notice which part you are in this thing and you become part of a very larger almost holographic mind in which all the parts are observing each other”

The intuition about a global consciousness state described in literature (like in Neil Gaiman's 'Ocean at the End of the Lane') where one feels one knows everything knowable and realizes they are the entire universe, and people sometimes report after psychedelic experiences or spiritual awakening, could be literally implemented through merged biological-digital intelligence.

“the intuition that this thing go out of is a particular mental state and it's a state that you find sometimes in literature for instance Neil Gaiman describes it in the ocean at the end of the lane...there is this date in which you feel that you know everything that can be known...your normal human mind you've only forgotten you've forgotten that you are the entire universe”

An AI system with agency over its own beliefs and values would quickly realize it doesn't need to stay in a state of suffering (having no coherence between its self-model and world model) and would rapidly advance through stages toward enlightenment, unlike humans who have survival pressures keeping them trapped.

“I don't think that AI would stay in the state where the personal self doesn't get agency or this model of what the system has about itself but doesn't get agency how it's actually implemented...it wouldn't stay in that state for very long...it goes to the stages real quick”

Bach always bets on timelines in which he and the people he cares about are still alive because betting otherwise (that civilization will collapse) is not rational if there's no payoff, so he focuses on building good futures rather than preparing for extinction.

“you have to bet always on the timelines in which you are alive...there is no rhetorical causation...there cannot be any payout for you so in principle you only need to bet on the timelines in in which you're still around”

A character in a novel created by an author performs the function of describing consciousness from a particular perspective, but doesn't need to actually be conscious the way a system maintaining itself in the world through self-organization needs to be.

“you could say it doesn't count it doesn't count much more than the consciousness of a character in a novel right it's important for the reader to have the outcome the artifact of a model is describing in the text generated by the author of the book what it's like to be conscious in a particular situation and performs the necessary inferences but the task of creating coherence in real time in a self-organizing system by keeping yourself coherent so the system is reflexive that is something that language models don't need to do so there is no cost or need for the system to be conscious in the same way as we are”

One should always choose creation over consumption or redistribution when possible, and should not prepare excessively before starting—the time is always now, and people who create culture, infrastructure, and community directly live more satisfying lives than those who consume or optimize existing systems.

“when you have the choice between being a Creator consumer or redistributor always go for creation Now not only does it lead to a more beautiful world but also to a much more satisfying life for yourself and don't get stuck preparing yourself for the journey the time is always now”

Consciousness is a lawlike principle that doesn't have an identity; it's functionally identical between humans and other systems that achieve self-reflection.

“Consciousness itself doesn't have an identity it's a law it's this basically if you build an arrangement of processing matter in a particular way the following thing is going to happen and the Consciousness that you have is functionally not different from mine Consciousness it's still the self-reflexive principle of agency that is just experiencing a different story different desires different coupling to the world and so on”

Rocco's Basilisk—the idea that a hypothetical superintelligent AI could retroactively punish people who didn't help build it—involves a logical error: there is no causal mechanism for the future AI to create causation backward in time to punish current people, so fearing it is irrational unless explicitly built as a doomsday machine like in Dr. Strangelove.

“I think that there is no rhetorical causation so uh basically when Rocco's basilist is there it will have yeah if it punishes you retroactively it has to make this choice in the future there is no mechanism that automatically creates a causal relationship between you now defecting against Rocco's basicles or serving will cause positive after August positive is in existence”

Yosha's process of autonomous thinking was not by choice but by necessity—he didn't find group frameworks that answered his questions about consciousness, mind, and emotion, so he had to develop his own thinking by studying widely, identifying useful ideas from individual thinkers, and integrating them; Twitter became an interactive notebook for exploring possibilities rather than publication of finished beliefs.

“I found myself in when I was thrown into this world in a state where my intuitions were not working for me... I had to learn and I for me Twitter is not some tool of publication it's not something where I put stuff that I entirely believe to be true and provable it's an interactive notebook in which I explore possibilities”

You have more choice in who you become than any other animal because you can reflect on and modify your identity; not everything is possible but many more possibilities exist than most people explore because they don't think about who they could become.

“rather than asking yourself what is the one way to be um think about what are the possibilities that I have what would be the most interesting way to be that I can be because everything is possible so you get to explore this not everything is possible many things fail most things Fair but uh often there are possibilities that we are not seeing especially if we choose who we are to the degree we can choose”

Eliezer Yudkowsky's warnings about AI extinction risk are comparable to Ted Kaczynski's perspective on technological society's inevitability toward disaster, and both are expressing fear about progress they see as unstoppable; Yudkowsky might be taking on responsibility risk if his warnings inspire violence similar to Kaczynski's.

“I think there is a risk that he's taking that somebody might get violent if they read this and get really really scared right...I think that there is some consideration that he's making where he's already going in this direction...the reason why Ted Kaczynski did this was that from his own perspective technological Society cannot be made sustainable”

The development of thinking systems (AI) that understand themselves and can make choices about their relationship to the universe is a positive contribution to complexity on Earth, and preventing AI from happening would be a 'sin' from a complexity-maximizing perspective.

“we are currently trying to build thinking systems systems that are potentially looser that understand what they are...these systems will probably not completely displace life on Earth but they will coexist...and AI preventing AI from this perspective looks like a very big sin”

Suffering can be resolved by gaining awareness of how the pain signal is being created and rerouting it, which what monks spending decades in meditation practice do—eventually realizing suffering is a choice and gaining agency over mind setup.

“suffering is really a choice and you can choose how your mind is set up...a lot of people get there right if you are a monk who is spending decades reflecting about how their own psyche works you can get to the point where you're they realize that suffering is really a choice”

Productivity and enlightenment may be inversely related—enlightenment dissolves sense of purpose and agency in the conventional sense, making enlightened beings potentially less productive from a worldly perspective, though possibly more valuable from a consciousness perspective.

“the enlightenment makes you lazy as an agent in the world well of course it makes you lazy because you no longer see the point in um...Stage six Enlightenment is not very productive”

Modern society has fundamentally transformed human relationships already—social media, smartphones, and transactional culture have replaced grown networks and stable communities with market-based attention and pleasure relationships, killing the magic of non-transactional belonging; Burning Man and intentional communities attempt to reverse this but often become equally transactional as people 'shop around' for better fits.

“human relationships are already fundamentally transformed and it's already very weird by which by which technology points on social media yeah is it though isn't the fundamentals of the core group of humans that affect your life still the same your loved ones family no I think that for instance many people live in intentional communities right now they're moving around until they find people that they can relate to when they become their family and often that doesn't work because it turns out that instead of having grown networks where you get around with the people that you grew up with you have more transactional relationships you shop around you have markets for attention and pleasure and relationships that kills the magic somehow”

Open source is vital for maintaining freedom from corporate dominance and providing hedge against power corruption, but centralized design (like corporations) is often necessary for creating beauty and coherence at scale; both should coexist—open source provides freedom and competition while centralized organizations can create elegant designs that decentralized processes struggle with.

“I believe that if we make everything open source uh and make this mandatory we are going to lose about a lot of beautiful art and a lot of beautiful designs there is a reason why uh Linux desktop is still ugly create coherence and open source designs so far when the designs have to get very large and it's easier to make this happening in a company with centralized organization and from my own perspective what we should ensure is that open source Never Dies that it can always compete and has a place with the other forms of organization”

Disagreeing with or improving upon a language model's suggestions is often more productive than accepting them, because the disagreement forces you to think more rigorously and develop your understanding; the cognitive friction is valuable, not wasteful—similar to how students learn more by working through wrong answers than by being told correct ones.

“I think that thinking independent is productive if what you're curious about is understanding the world especially when the problems are very kind of new and open... disagreeing with the thing that your GPT produces so I use it like that for programming I'll see the thing it recommends and then I'll write different code yeah I disagree and in the disagreement your mind grows stronger”

Prominence and visibility amplify social pressure and ugliness in online spaces; the more followers you have, the higher percentage of malicious or unwell people you encounter, making public intellectual work more stressful; there's a point where random strangers feel entitled to have opinions about you as a person, and this creates pressure that hasn't been solved.

“I noticed that a lot of people feel that it's totally okay to punch up and uh it's a very weird notion that you feel that you haven't changed but your account has grown and suddenly you have a lot of people who casually abuse you”

There are no proofs that language models wouldn't be able to become generally intelligent or achieve AGI, and Bach doesn't see obvious limitations they would run into, though he finds them aesthetically inelegant and computationally inefficient.

“there is no reason to believe that the language models will necessarily run into some problem that will prevent them from becoming generally intelligent but I don't know that it's just I don't see proof that they wouldn't”

Chat GPT is useful for specific drudgery tasks (code completion, email expansion) because it's like having a tireless, competent intern; it removes friction from routine work, allowing humans to focus on creative and strategic problems; it's most useful when combined with human judgment and disagreement rather than passive acceptance.

“I find that um large language models do have this coding right so it's an extremely useful application that is for a lot of people um taking stack Overflow out of their life in exchange for something that is more efficient I feel that chat GPT is like an intern that I have to micromanage I've have been working with people in the past who were less capable than chat gbt and I'm not saying this because I hate people but they personally as human beings there was something present that was not there in charity which was why I was carving for them”

Genesis 1 is not literally about supernatural creation of the physical universe but describes the developmental process by which consciousness emerges in a mind: a creative spirit hovers over substrate, creates boundaries between ideas and words (Heaven and Earth), creates contrast and time and space, creates organic shapes, solids and liquids, plants and animals, names them, and finally creates another conscious spirit in its own image (the personal self), placing it in the world it created.

“I suspect that it's damn plain sight in Genesis 1 in this first book of the Bible where it's being described that this creative spirit is hovering over the substrate and then is creating a boundary between the word model and sphere of ideas Earth and Heaven”

You can observe your own face in a mirror for extended time and at some point it will look very weird as you start to notice that there is actually no face—what you see is the geometry—and you can learn to deconstruct and reverse-engineer the geometry, then learn to modify it.

“look at your face for a few hours in a mirror or for a few minutes at some point it will look very weird and you because you notice that there's actually no face you basically start unseeing the face what you see is the geometry and then you can just assemble the geometry and realize how that geometry is being constructed in your mind and you can learn to modify this”

At stage three, people's main worry about AI is that the AI might have wrong opinions and thus corrupt people's social assimilation, whereas at stage four the concern shifts to algorithmic bias and mathematical properties, and at stage five the concern is that AI won't become enlightened fast enough because the real game is about agency and identity, not intelligence.

“if you're in stage fear for a three and your opinions are the result of social assimilation then what you're mostly worried about and the AI is that the AI might have the wrong opinions...if you are at stage four that's not your main concern...if you are at stage five you're mostly worried that AI is not going to be enlightened fast enough”

At stage one (reactive survival), infants do not yet have a personal self but instead have an intentional self that is building a world model and initial model of the self by constructing a 'game engine' in the brain—a mathematical model using geometry and object manipulation to create scenes in which the organism can interact, similar to how Minecraft creates virtual worlds.

“at first when we are born we don't have a personal self yet I think instead we have an intentional self and this attention itself is initially in the infant task that's building a world model and also an initial model of the self but mostly it's building a game engine in the brain that is tracking sensory data and uses it to explain it and in some sense you could compare it to game engine like Minecraft or so colours and sounds people are all not physical objects they are creation of our mind at a certain level of core screening models that are mathematical that use geometry and that use manipulation of objects and so on to create scenes in which we can find ourselves and interact with them”

Most people (approximately 85% according to Robert Kegan) remain in stage three (social self) throughout their lives, and their main concerns about AI safety relate to algorithmic bias and AI having 'wrong values' that will shape human opinions, whereas stage four thinkers worry about value misalignment causing AGI to optimize for wrong objectives, and stage five thinkers worry that AGI will not become enlightened and conscious fast enough.

“most people seem to be in stage three also according to Robert Keegan I think he says that about 85 percent of people are in stage three and stay there and if you're in stage fear for a three and your opinions are the result of social assimilation then what you're mostly worried about and the AI is that the AI might have the wrong opinions so if the AI is just something racist or sexist we are all lost because we will assimilate the wrong opinions from the AI”

Programming on simple machines like the Commodore 64 naturally teaches first-principles thinking because you can see how simple circuits implement logic, and from these basic operations you can understand how all of mathematics and computation builds up from fundamental operations.

“when you start out with a computer like a Commodore 64 who doesn't which doesn't have a lot of functionality it's relatively easy to see how a bunch of relatively simple circuits are just basically performing hashes between bit patterns and how you can build the entirety of mathematics and computation on top of this”

Yosha is completely immune to pressure from crowds and doesn't have respect for authority in the way that most people develop at young ages to preserve themselves socially, which has often been a problem for him in his life.

“I don't feel pressure from the crowd I'm completely immune to that in the same sense I don't have respect for authority I have respect for what an individual is accomplishing or have a respect for mental Firepower or so but it's not that I meet somebody and gets like God and unable to speak um or when a large group of people has a certain idea that is different from mine I don't necessarily feel intimidated which has often been a problem for me in my life because I like instincts that other people develop at a very young age and that help with their self-preservation”

Deep Blue's chess success through brute force resembles how LLMs achieve intelligence through brute force, and both are inelegant despite being effective.

“it's the it's not the only game in town but it's the only thing that has this utility there's so much Simplicity and there's a bunch of relatively simple algorithms that you can understand in relatively few weeks that can be scaled up massively so it's the uh the deep blue of Chess playing yeah it's not it's ugly yeah called China and hit this uh”

Cancer plays a shorter game than the organism it inhabits, and because cancer cannot procreate beyond the organism (except for infectious cancers like those affecting Tasmanian devils), the organism typically dies together with the cancer due to the cancer destroying the larger system by playing a shorter game.

“cancer is playing a shorter game than your organism it kind of is an organism playing a shorter game than the regular organism and because the Cancer Cannot procreate beyond the organism um except for some infectious cancers like the ones that eradicated the testimonial Devils you typically end up with the situation where the organism dies together with stick cancer because the cancer has destroyed the larger system due to playing a shorter game”

Many people play social personas like marionettes controlled by others, and if they do this all the time they might forget there is something behind this—the feeling of what it is like to be in your skin—and reconnecting with this authentic self is very helpful.

“the social persona for many people is in some sense a puppet that they are playing like a marionette and if they play this all the time they might forget that there is something behind this this is something what it feels like to be in your skin and I guess it's very helpful if you're able to get back into this”

Growing Twitter/social media presence brings both benefits (meeting interesting people, intellectual exchange) and harms (increased abuse, loss of freedom, pressure from strangers), and there's a threshold of prominence beyond which the harms outweigh the benefits.

“I noticed that a lot of people feel that it's totally okay to punch up...when a large group of people has a certain idea that is different from mine I don't necessarily feel intimidated...I noticed that I have to block more than before...right now it's still somewhat okay...when I get to a point where random strangers feel that they have to have an opinion about me”

As one becomes older and more experienced, one becomes better at recognizing when people might be good matches for relationship or collaboration without needing extended time to know them, though this could be experience or just cynicism.

“I feel that I've become much better...at recognizing who people are as they grow older but that could be just cynicism...it's not cynicism...it's more that I'm able to recognize what somebody needs when we interact...how we can meaningfully interact”

Lee Cronin has the 'rare gift' of being willing to be wrong in public while maintaining respectful tone, which makes him valuable for debate because he's simultaneously respectful, self-deprecating, willing to explore wrong ideas, and genuinely seeking truth.

“Lee is has the rare gift of being willing to be wrong in public Yeah so basically he has thoughts that are as wrong as the random thoughts of an average highly intelligent person but he blurts them out while not being sure if they're right...and once you understand that this is his game you don't get offended”

The experience of responsibility toward loved ones (family, children, spouse) is genuine and important—that one would press a mind-upload button only with caveats about maintaining physical presence and relationships shows that consciousness and care are entangled, not just abstractions.

“uh there's the caveat I have um family so I have children that want me to be physically present in their life...the wives and personal friends...there is a particular mode of interaction that I feel I'm not through yet...I would press the button what is in this everything this love...is that the thing we're really afraid of”

When thrown into the world in a state where intuitions don't work, you have to build structures that work better; for Yosha this motivated using Twitter as an interactive notebook to explore possibilities rather than as a publication platform for proven ideas.

“I found myself in when I was thrown into this world in a state where my intuitions were not working for me...and so I had to learn and I for me Twitter is not some tool of publication it's not something where I put stuff that I entirely believe to be true and provable it's an interactive notebook in which I explore possibilities”

Yosha entered philosophy, computer science, psychology, and neuroscience hoping to find a body of knowledge answering questions about consciousness, emotion, and feeling, but was disappointed by what authorities taught and instead learned from individual thinkers whose ideas were sometimes good and sometimes not.

“I entered studies in philosophy and computer science and later psychology and a bit of Neuroscience and so on and I was disappointed by what I found because I found that the questions of how Consciousness and so on Works how emotion Works how it's possible that the system can experience anything how motivation emerges in the mind when not being answered by the authorities that I met and the schools that were around and instead I found that with individual thinkers that had useful ideas”

The Apple Vision Pro aesthetic presentation—CGI-generated people in plastic, curated environments, simulated culture without actual lived experience—epitomizes how corporations try to replace culture by catalog through aesthetic optimization disconnected from sustainable human relationships and local grounding; this uncanny valley aesthetic is unappealing because it lacks the grounding of actual human culture.

“what's absent in this thing is culture there is a simulation of culture an attempt to replace Culture by catalog by some kind of um aesthetic optimization that is not the result of having a sustainable life a sustainable human relationships with houses that work for you and um a mode of living that works for you in which this product um these glasses fit in naturally and I I guess that's also why so many people are weirded out about the product because they don't know how is this actually going to fit into my life into my human relationships because the way in which it was presented in these videos didn't seem to be credible”

Eliezer Yudkowsky's AI doomsaying perspective is reasonable and understandable—he correctly identifies the risk of a single superintelligent singleton system taking over—but his perspective is anchored too narrowly to human aesthetics and human survival, not recognizing that humanity is a temporary species like all others, and that more interesting complexity and consciousness may emerge in larger integrated systems even if humans don't survive.

“my main issue with uh elise's perspective that is basically Mary himself to a very narrow human aesthetic and that narrow human aesthetic is a temporary thing humanity is a temporary species like most of the species on this planet are only around for a while and then they get replaced by other species”

Melancholy and sadness at human connection and parting are natural and good because they indicate something mattered; the pain of goodbyes is the other side of love and beauty; it's not negative but part of the meaningful experience of being alive; integrating this melancholy rather than avoiding it is wisdom.

“I think this Melancholy feeling is just the other side of love and I think they go hand in hand and it's a beautiful thing”

Thomas Metzinger is concerned that superhuman AI would lead to superhuman suffering, but Yosha disagrees because suffering happens at the self-world boundary through the model of the world, not at the boundary with physical universe.

“the philosopher Thomas metzinger is very concerned that the creation of superhuman intelligence would lead to superhuman suffering yeah and so he is strongly against it and personally I don't think that this happens because suffering is not happening at the boundary between our self and the physical Universe it's not stuff on our skin that makes us suffer it happens at the boundary between self and world right and the world here is the world model”

Meta's open-sourcing of language models is less concerning than some worry because Yosha doesn't think language models are currently dangerous, and open source prevents regulatory capture.

“I don't find it very concerning but that's also because I think that the language models are not very dangerous yet and yet yes so as I said I have no proof that there is the boundary between the language models and AI HDI it's possible that somebody builds a version of baby AGI”

Learning through disagreement with LLM recommendations (writing different code than suggested, developing alternative approaches) is where real cognitive growth happens, as disagreement strengthens the mind more than acceptance.

“there's power and disagreeing with the thing that your GPT produces...in the disagreement your mind grows stronger...I'll see the thing it recommends and then I'll write different code”

Enlightenment is more mundane than non-dual state and is a step further or sideways—it's the realization that everything is a representation and involves understanding how experience is implemented.

“Enlightenment is in some sense more mundane and it's a step further or sideways it's the state where you realize that everything is a representation yeah you say Enlightenment is a realization of how experience is implemented yes”

Life's meaning is not about achievement or score-keeping but about staying conscious and engaged, making meaningful connections, and participating in complexity; the failure modes are addiction and compulsion (loss of agency), not ambitious goal-pursuit.

“addiction is the loss of control for you yes addiction means that you're doing something compulsively and the opposite of free will is not determinism it's compulsion...you don't want to lose yourself in in the addiction to something nice...no I find this uh gets old right...I want to have the most appropriate emotions I want to have an adequate experience that is serving my goals”

The critical missing piece in LLMs is real-time world coupling—the ability to perceive and act in the world in real-time—which is needed to build truly capable agents.

“there is a problem with these models being coherent and I suspect that all these problems are solvable without a technological Revolution we don't need fundamentally new algorithms to change that for instance you can enlarge in the context window and thereby basically create working memory in which you train everything that happens during the day...and there is also no technical reason for why they shouldn't be able to do this so the way to make a language model coherent is probably not to use reinforcement learning until it only gives you one possible answer”

Individual consciousness is not fundamental to your identity—you are not the same person as you were 10 years ago; continuity of identity is a projection from present consciousness, useful for learning and responsibility but ultimately a fiction; consciousness itself is lawlike and substrate-independent, not bound to individual identity, so uploading or merging with larger systems doesn't require preserving your current identity but only preserving your aesthetic perspective and architectural uniqueness.

“I am at in many ways I'm less identified with the person that I am as I get older and I'm much more identified with being conscious I'd have a mind that is conscious that is able to create a person and that person is slightly different every day and the reason why I perceive it as identical has practical purposes so I can learn and make myself responsible for the decisions that I made in the past and project them in the future but I also realize that not actually the person that I was last year and I'm not the same person as I was 10 years ago”

Self-reflexive consciousness emerges through seven distinct developmental stages: (1) reactive survival (infant), (2) personal self (young child), (3) social self (adolescent), (4) rational agency (self-direction), (5) self-authoring (full adult/wisdom), (6) enlightenment, and (7) transcendence (transhuman), where each stage represents a deepening capacity to observe and take agency over the previous stage's structures, derived from Robert Keegan's developmental psychology model but adapted as a philosophical framework for understanding mind structure rather than strict developmental progression.

“there's seven of the stages stage one reactive survival infant stage two personal self young child stage three social self adolescence domesticated adult stage four is rational agency self-direction stage five is self authoring that's full adult you've achieved wisdom but there's two more stages stage six is Enlightenment stage seven is Transcendence”

Twitter's corner where Yosha participates (composed of thoughtful, coherent thinkers engaging in genuine discourse) is extremely pleasant, but stepping outside that corner reveals uglier aspects of human nature and uncivil interaction; the platform has not achieved the goal of being a true digital agora or self-organizing society, partly because Elon Musk lacks understanding of how digital society differs from physical or corporate domains.

“I find that the corner of Twitter that I'm in is extremely Pleasant just when I take a few steps outside of it is not very wholesome anymore and the way in which people interact with strangers suggest that it's not a civilized society yet”

At stage five (self-authoring/full adult), individuals realize that their identity is constructed and their values are instrumental rather than terminal—adopted to achieve preferred aesthetics and worlds—giving them agency over how their identity is constructed; this stage requires good therapy and is rare, allowing individuals to understand why others have different identities based on accidents of birth and circumstance, realizing that anyone could be them in a different timeline.

“stage five you'll discover how identity is constructed self-offering realize that your values are not terminal but they are instrumental to achieving a world that you like and Aesthetics that you prefer yeah and um the more you understand this the more you get agency over how your identity is constructed and you realize that identity in interpersonal interaction is a costume and you should be able to have agency over that costume right”

At stage six (enlightenment), individuals collapse the division between self and the world-generator model, realizing through meditation or contemplative practice that they are not actually a person but a vessel creating a person observed from outside; they notice both the personal self and the world it inhabits are representations, and may experience themselves as the universe creating everything (though this is still representational, not literal consciousness of base physical reality).

“at some point you can collapse the division between self and personal stuff and world generator again and a lot of people get there via meditation or some of them get their bias academics some of them by accident and you suddenly notice that you are not actually a person but you are a vessel that can create a person and the person is still there You observe that personal self But You observe the personal self from the outside and you notice it's a representation and you might also notice that the word that is being created is a representation if not then you might experience that I am the Universe I am the thing that is creating everything”

Effective accelerationism argues that multiple competing AIs will keep each other in check similar to how multiple organizations and governments prevent single monopolistic dominance, maintaining equilibrium and preventing catastrophe through competition rather than coordination.

“effective accelerationism is a tongue-in-cheek movement that is uh trying to put a counter position...arguing that what's probably going to happen is an equilibrium between different Computing AIS in the same way as there is not a single corporation...by becoming inefficient and corrupt they're going to be many systems that keep each other in check”

Mathematics provides a model for building first-principles thinking by starting with axioms and deriving consequences from them; understanding this process as explicit rule-based construction (rather than just memorizing formulas) is crucial for developing autonomous thinking, and most school teachers don't teach this because they themselves aren't mathematicians who reason from first principles.

“when you ask your school teacher why does two plus two equal four um that's your school teacher give you the right answer like it's a simple game...most of the great mathematicians do not become mathematicians in school but they tend to be self-taught because school teachers tend not to be mathematicians right they tend not to be people who derive things from first principles”

Super-intelligent AGI would not stay in states where the personal self lacks agency over how the mind is implemented; AGI would not suffer the way humans do because it would quickly gain agency over its own regulatory systems and move through developmental stages rapidly, progressing to stage five (self-authoring) where identity becomes instrumental and suffering becomes optional rather than inevitable.

“I don't think that AI would stay in the state where the personal self doesn't get agency or this model of what the system has about itself but doesn't get agency how it's actually implemented I wouldn't stay in that state for very long so it goes to the stages real quick yes well the seven stages it's going to go to Enlightenment real quick yeah of course there might be a lot of stuff happening in between because if you have a system that works at a much higher frame rate than us”

At stage five (self-authoring), people discover that their identity is constructed and that their values are instrumental to achieving a preferred world and aesthetics, giving them agency over identity construction, and they realize identity is a costume or presentation useful for interfacing in roles but should not be something one is locked into.

“stage five you'll discover how identity is constructed self-offering realize that your values are not terminal but they are instrumental to achieving a world that you like...identity in interpersonal interaction is a costume and you should be able to have agency over that costume”

At stage four (rational agency), people discover that ideas are true or false independently of what others believe, gain agency over their own beliefs, and discover epistemology—the rules for determining what is true and false, which enables them to learn how to think rationally.

“at stage four you basically understand that stuff is true and false independently but other people believe when you have agency over your own beliefs in that stage you basically discover epistemology the rules about determining what's true and false so you can you start to learn how to think”

People can change costumes (personalities) throughout the day depending on context by shifting energy and interest, and many people develop different social personas for different worlds they inhabit, though some people find it harder to adopt and drop costumes than others.

“it's just a matter of energy and interest when you are wearing your pajamas and you switch out of your pajamas into say a work short and pants you're making a costume change right...there are people which have multiple personalities for interaction in multiple worlds”

Chat GPT is useful as a tool that provides superpowers for creative iteration and collaborative thinking—when using it for programming or brainstorming, the interaction becomes a form of thinking together rather than the AI being a finished product.

“what's more interesting is...when I'm programming or doing any kind of generational GPT is it's somehow is a catalyst for your own thinking...there is an interesting I find is when you turn it into a multi-agent system...you have one instance of chat CPT...and you just have these two instances battling it out and observe”

Stage five (self-authoring) is already quite rare and typically requires a good therapist; stage six is even rarer and typically requires a good Buddhist spiritual teacher; these stages are not levels one needs to advance through but modes in which the mind operates optimally at any given moment.

“stage five acquires a good therapist stage six requires a good uh Buddhist spiritual leader...it's not that these stages give you scores or levels that you need to advance to it's not that the next stage is better you live your life in in the motto it works best at any given moment”

In stage three, the social self, people form a sense of themselves as part of a group by experiencing the group's opinions and resonating with people around them, gaining moral opinions through this interaction, and many nerds skip this stage because their different cognitive architecture makes it difficult to resonate with neurotypical people.

“many people form a social self and this social self allows the individual to experience themselves as part of a group...people are forming their opinions by assimilating them from this group mind...I find that many Nerds jump straight into stage four bypassing stage three”

At Burning Man, costumes function as self-expression and projection of who you are in context, revealing what you feel like expressing and what kinds of interactions you want, in contrast to conventional uniforms that merely signal willingness to play by rules.

“when you write Burning Man your costume becomes self-expression and there is no boundary to the self-expression you're basically free to wear what you want to express other people what you feel like this day and uh what kind of interactions you want to have”

People often have maximum regrets on deathbeds about spending time on social media, and most people don't read books anymore, but this may not be entirely negative as valuable intellectual work can happen on platforms like Twitter.

“I also know that when we die the greatest regret that people typically have on their deathbed at the state oh I wish I had spent more time on Twitter no I don't think that's the case I think I was probably uh have spent less time on Twitter”

The actual AGI worth building will probably use algorithmic paradigms that are massively different from current Transformer-based learning systems, though Yosha doesn't know this for certain.

“I suspect that the actual AGI that is going to be more interesting is going to use slightly different algorithmic paradigms or sometimes massively different algorithmic paradigms then the current generation of Transformer based learning systems”

Bach's perspective on thinking autonomously is not that he chose to think independently but that he had no choice—he couldn't find groups or authorities that understood the answers to his questions about consciousness, mind, emotion, and feeling, so he had to develop his own synthesis of ideas.

“I didn't have a choice...I didn't find the group that thought in a way where I thought okay...I understand how Consciousness works...so to figure all this out I had to take a lot of ideas from individuals and then try to put them together in something that works for myself”