Positional encoding uses sine and cosine functions where for every odd time step a cosine vector is created and for every even time step a sine vector is created, then these vectors are added to their corresponding embedding vectors to give the network position information.
factualpending
Speaker
Unidentified Speaker — Illustrated Guide to Transformers Neural Network: A step by… [4Bdc55j80l8]Evidence Quote
“for every odd time step create a vector using the cosine function for every even time step create a vector using the sine function then add those vectors to their corresponding embedding vector”
Created: 8/13/2026, 9:51:15 AM
My Notes
Loading notes...