Glossary

Multi-query attention

A form of multi-head attention in which all the heads share a single set of keys and values. It makes generation faster because much less data must be loaded from memory at each step.