AlexNet was trained on 2 GPUs, the original Transformer paper used 8-64 GPUs maximum, and o1 reasoning was not the most compute-heavy thing, demonstrating that research breakthroughs do not require the absolute maximum available compute, only sufficient compute to validate ideas.