The most plausible path to beneficial outcome is first AGI created by group wanting to make it open and decentralized, not controlling it personally, not imposing their own value system, then AGI rapidly makes plan for secure beneficial development and rolls out globally.

The bar for AI adoption is high: AI must be way way cheaper or better than people, and when margin is large enough, social obstacles to adoption will be overcome—it's a cost-benefit calculus.

If you train an LLM on all music up to 1900 without data after 1900, it will never invent neoclassical metal, grindcore, progressive jazz, hip-hop—it won't synthesize new genres the way humans did.

People are not well aligned with themselves, let alone with each other—we're clusters of behavior patterns rather than unified, rational, coherent entities when honest with ourselves, and that's even more true at the collective level.

Without concrete, visible demonstrations of capability (like ChatGPT), most political decision-makers and CEOs will not take serious things seriously—they require tangible evidence in front of them rather than projections or imagination of future possibilities.

Invention often happens through cobbling together available materials to make something work, and the most elegant simple Occam's Razor way often comes later—mathematical proofs are often initially ugly messes that get simplified decades later, as was true for early quantum mechanics versus Heisenberg and Schrödinger's formulations.

Law is another area where AI could do the job better and cheaper—LLMs can do parallegal work, contract drafting—but legal profession is in no hurry to restructure because of licensing protections and monopoly preservation.

One meaningful approach to alignment is minimal commitments: you want AGI to not destroy humanity, not cause human extinction, and not be in complete control over humanity—these are points of wide agreement.

US AGI won't want to go slowly if China's AGI goes fast, and decentralized network building AGI won't vote to go slowly if centralized network is going fast—arms race psychology creates prisoner's dilemma for responsible AGI development.

Robust embodiment is convenient but probably not necessary—you could get vastly superhuman AGI with much restricted sensorium and motoric world, especially for theorem proving and scientific research using symbol manipulation.

A non-human AGI will have less understanding of human values, culture, and what it is to be human—having something resembling human embodiment would give it a head start in relating to us on a deep level, even if it doesn't guarantee empathy.

Whether job displacement comes gradually or in wave might depend more on face transition dynamics of human social networks making up the economy than on AI capability itself.

The slowdown factor from AGI to ASI will be the AGI's own value system—even if I could rewrite my own brain arbitrarily fast, I probably wouldn't because I want to survive and care about mental well-being of family/friends.

Making systems provably secure is much more expensive—my secure transformer slows things by factor of 2x, and homomorphic encryption slows by several hundred times, making it unfeasible for current AI development.

Gary Marcus is correct that some people oversell LLMs and they have limits to general intelligence, but society could reorganize to adopt AI for most jobs if we had political will to give people free money and not require work.

From where I stand now, most responsible thing is not to go from AGI to superintelligence at maximum possible speed—probably best is to go gradually using experimental information gathering—but this requires trust among competing AGI parties.

At age 7 or 8 in 1973, I found a book by Princeton physicist Gerald Fineberg called 'The Prometheus Project' that laid out a coherent argument that within the next few decades we would get machines smarter than people, strong nanotech to build machines from molecules, and the ability to fix aging—this convinced me AGI was possible in my lifetime.

By the late 1960s, it was already mathematically and conceptually clear from the work of Turing, McCulloch-Pitts, and others that brains could be viewed as computing devices, computers were getting faster and faster, and by the early 1970s we had already beaten the world champion of checkers—so the theoretical foundations for AGI were already established.

The impact of scale has been the main factor in the current AI revolution—scale of compute and data allowed the LLM revolution to happen, and this same primary underlying factor will let AGI happen.

When I was teaching deep neural networks in the mid-1990s at University of Western Australia, training a multi-layer perceptron with just 35 neurons on a fast Sun workstation took three hours, making refinement and experimentation infeasible—now those same experiments run in minutes.