A transformer network is a type of neural network architecture used by large language models that includes an embedding layer (turning words into patterns of numbers), attention layers (computing interactions among words), and traditional neural networks, arranged in multiple layers (GPT-3 has about 100 transformer blocks pipelined together).