Grouped-query attention (GQA)

A form of multi-head attention in which the heads are split into groups, and each group shares one set of keys and values. It is nearly as fast as multi-query attention while keeping nearly all the quality of standard attention.