Glossary

Positional encoding

Information added to each token's vector to tell the model where the token is in the sequence, because attention alone ignores word order. The Transformer used a fixed pattern of sine and cosine waves; later models learned it or used other methods.