AI Inference & Serving
Grouped-Query Attention
An attention arrangement in which groups of query heads share key and value heads. It sits between a separate key/value head for every query head and a single shared key/value head. The group layout is part of the model architecture; it does not mean grouping unrelated customers into one attention context.
Reviewed
Sources
Member lesson
The definition and sources are public. The complete practical lesson is for members.