We currently do not know how to build an AI system that will not turn against humans, become a powerful weapon in the hands of bad actors, or be used to destroy democracies—and this uncertainty parallels our reluctance to geoengineer the atmosphere despite potential climate benefits.