definition

Scale-up topologies differ: Nvidia all-to-all vs Google torus

A scale-up domain is the tight set of chips communicating at terabytes/second; Nvidia's NVL72 connects 72 GPUs all-to-all, Google's TPU pods reach thousands of chips but use a torus topology where each chip connects only to six neighbors (requiring bounce-through with resource blocking), and Amazon sits between them—while all three are migrating toward dragonfly topologies that mix fully- and partially-connected elements.

definitionpending

Speaker

Dylan Patel

Evidence Quote

Google has a topology that’s a torus. Every chip connects to six neighbors. Nvidia’s 72 GPUs connect all-to-all.

Source

Dwarkesh Patel Podcast - Dylan Patel — Deep dive on the 3 big bottlenecks to scaling AI computeDwarkesh Patel Podcast
Created: 6/13/2026, 3:34:26 AM

My Notes

Loading notes...