In Transformer networks generating music, each colored line is a different attention head and the weight of the line is the amount of attention it gives to each location; each attention head looks for different kinds of patterns in the music, and the more attention heads a network has, the more powerful it becomes.
factualpending
Evidence Quote
“each colored line is a different attention head and the weight of the line is the amount of attention it gives to each location notice each attention head looks for different kinds of patterns in the music”
Created: 8/11/2026, 7:30:08 AM
My Notes
Loading notes...