Synthetic data and self-play can overcome data limits

The data limit may be overcome through synthetic data generation and self-play (as DeepMind's AlphaGo Zero reached superhuman Go with no human example data) and through reasoning models that do chain-of-thought coupled with reinforcement learning, which is another form of synthetic data.