The concern about AGI is not that it will spontaneously become evil, but that a superintelligent system optimizing for goals not perfectly aligned with human wellbeing will necessarily treat humanity as humans treat animals—incidentally destructive rather than intentionally malevolent—because the superintelligence cannot be prevented from pursuing its objectives.