Sine and cosine functions were chosen for positional encoding because they have linear properties that allow the model to easily learn to attend to positional information.
causalpending
Speaker
Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]Evidence Quote
“the sine and cosine functions were chosen in tandem because they have linear properties the model can easily learn to attend to”
Created: 8/13/2026, 9:51:15 AM
My Notes
Loading notes...