Sebastian Bubeck's team has trained a 1 billion parameter model entirely on synthetic data (never seeing any internet text) that produces safer outputs than models trained on internet text, suggesting it's possible to have capability and safety simultaneously through careful data generation.