The concern that post-human superintelligence might lack consciousness or inner experience (the 'lights out' scenario) is naive because all of the properties humans have—consciousness, qualia, subjective experience—exist because they are useful for achieving goals, so any sufficiently intelligent system pursuing goals will necessarily develop these properties.
causalpending
Speaker
Richard SuttonEvidence Quote
“all those properties we have... are valuable... useful for achieving goals... It's all just comes from intelligence”
Source
Richard Sutton - Humanity Never Had Control in the First Place (Worthy Successor Series, Episode 2)— The Trajectory with Dan FaggellaCreated: 8/12/2026, 6:16:37 PM
My Notes
Loading notes...