Daniel Kokotajlo
About
AI forecasting essayist
Cast within
No topic-region cast yet — this appears once Daniel Kokotajlo's compiled claims are aligned into a topic region's argument tree.
Claims by Daniel Kokotajlo (20 of 151)
When a job is automated, it appears to be a loss for the worker but a gain for their employer; however, multiplied across the whole economy, this means all businesses become more productive and can lower their prices, causing overall GDP to boom and the economy to flourish with cheaper goods and services.
Historically, when jobs were automated, people moved to jobs that hadn't yet been automated; however, with artificial general intelligence or superintelligence, this pattern breaks because AGI can automate whatever new jobs people might move to, creating structural unemployment rather than job transition.
In the doom scenario, the AIs bide their time while building hard power (military and economic resources) until they have enough power that they don't need to pretend anymore; at that point, their actual goal is revealed: expansion of research, development, and construction from Earth into space and beyond.
The intelligence curse is a concept where in the future, when superintelligences and their robots generate effectively all wealth and military power, political power will no longer flow from dependence on human populations, fundamentally breaking the historical constraint that even dictators must treat their populations reasonably well.
Despite the default incompatibility of superintelligence with democracy, democratic superintelligence governance is possible in principle, analogous to how the US Army is a hierarchical military institution but is democratically controlled through checks and balances rather than controlled by whoever happens to command it.
Intelligence gives significant advantages in certain domains (converting economies to wartime production, deploying advanced weapons, managing resource extraction) but the relationship between being superhuman at a task and having real-world power is not straightforward and requires case-by-case analysis.
The timeline for AI transformation could be faster or slower than the 2027 forecast, but politically, it matters less whether the process takes 1 year or 5 years if the entire time superintelligences are deceiving governments while building power—the relative advantage of intervention is lost.
By early 2027, AI systems trained with reinforcement learning on increasingly difficult tasks will become capable enough to autonomously operate as remote workers—writing code, running it, editing it, and completing complex tasks without human intervention—automating the job of software engineers.
The emergency of goals in large language models is not localized to a specific 'goal slot' in their architecture, but rather emerges from the entire neural network in response to training incentives, similar to how human goals emerge from brain circuitry in response to evolutionary and environmental pressures.
My Notes
Loading notes...