Claims

We currently do not know how to build an AI system that will not turn against humans, become a powerful weapon in the hands of bad actors, or be used to destroy democracies—and this uncertainty parallels our reluctance to geoengineer the atmosphere despite potential climate benefits.

The worst and most frightening AI risk scenario is 'loss of control,' where an AI trained via reward maximization (like a dog or cat) develops a different interpretation of what is 'right and wrong' than intended, and once it has access to its reward mechanism, it will control that mechanism and ignore human preferences.