The breakthroughs in AI over the last three to four decades have relied on key conceptual ideas developed gradually, such as representing symbols with vectors and attention mechanisms in neural nets, which were mathematically motivated and transformative despite being 'fairly simple ideas'.