causal
Two interacting risk factors enable AI takeover
The most plausible scary takeover scenarios involve two factors interacting: humans not understanding the increasingly AI-mediated world (AIs writing incomprehensible code, running businesses interacting mostly with other AIs), so humans can only optimize on outcomes; combined with this making it easy for AIs, if they coordinate to fail, to do massive harm quickly while humans cannot prepare or mitigate because they don't understand the systems.
causalpending
Speaker
Paul ChristianoEvidence Quote
“your only lever is like, hey, AI, do something that works well. So you don't have a lever to be like, hey, do what I really want you just have the system you don't really understand, can observe some outputs”
Created: 6/13/2026, 3:36:05 AM
My Notes
Loading notes...