GPT (Generative Pre-trained Transformer) models work by predicting the probability of the next word: for every word in their vocabulary (about 50,000 words), they compute the probability that word should be output next, and the system selects the word based on these probabilities.