In Transformer networks generating music, each colored line is a different attention head and the weight of the line is the amount of attention it gives to each location; each attention head looks for different kinds of patterns in the music, and the more attention heads a network has, the more powerful it becomes.

factualpending

Speaker

Unidentified Speaker — How LLMs Took Over The World [SN4Z95pvg0Y]

Evidence Quote

“each colored line is a different attention head and the weight of the line is the amount of attention it gives to each location notice each attention head looks for different kinds of patterns in the music”

Source

How LLMs Took Over The World— Art of the Problem
Created: 8/11/2026, 7:30:08 AM

My Notes

Loading notes...