Definition of the Alignment Problem

The superintelligence alignment problem is the problem of building a very powerful AI that steers the world toward where its creators wanted it to go; on a smaller scale today it is about getting an AI whose output and behavior match what programmers had in mind, distinct from whether the behavior is morally good.