grouped-query attention (gqa)
**Grouped-Query Attention (GQA)** is an **attention architecture that provides a tunable middle ground between Multi-Head Attention (MHA) and Multi-Query Attention (MQA)** — using G groups of KV heads (where each group serves multiple query heads) to achieve near-MQA inference speed with near-MHA quality, making it the recommended default for new LLM architectures as adopted by Llama-2 70B, Mistral, Gemma, and most modern open-source models.
**What Is GQA?**
- **Definition**: GQA (Ainslie et al., 2023) partitions the H query heads into G groups, with each group sharing a single set of Key and Value projections. When G=1, it's MQA. When G=H, it's standard MHA. Values in between provide a configurable quality-speed trade-off.
- **The Motivation**: MQA (1 KV head) is very fast but shows quality degradation on complex reasoning tasks. MHA (H KV heads) preserves quality but has an enormous KV-cache. GQA finds the sweet spot — typically 8 KV groups for 64 query heads gives ~95% of MHA quality at ~90% of MQA speed.
- **Practical Default**: GQA has become the de facto standard for new LLM architectures because it provides the best quality-speed Pareto curve.
**Architecture Visualization**
```
MHA: Q₁ Q₂ Q₃ Q₄ Q₅ Q₆ Q₇ Q₈ (8 query heads)
K₁ K₂ K₃ K₄ K₅ K₆ K₇ K₈ (8 KV heads — one per query)
GQA: Q₁ Q₂ Q₃ Q₄ Q₅ Q₆ Q₇ Q₈ (8 query heads)
K₁ K₁ K₂ K₂ K₃ K₃ K₄ K₄ (4 KV groups — shared pairs)
MQA: Q₁ Q₂ Q₃ Q₄ Q₅ Q₆ Q₇ Q₈ (8 query heads)
K₁ K₁ K₁ K₁ K₁ K₁ K₁ K₁ (1 KV head — shared by all)
```
**KV-Cache Comparison**
| Method | KV Heads | KV-Cache Size | Memory vs MHA | Quality vs MHA | Speed vs MQA |
|--------|---------|--------------|---------------|----------------|-------------|
| **MHA** | H (e.g., 64) | H × d × seq_len | 1× (baseline) | Baseline | Slowest |
| **GQA-8** | 8 | 8 × d × seq_len | 1/8× = 12.5% | ~99% | ~90% of MQA |
| **GQA-4** | 4 | 4 × d × seq_len | 1/16× = 6.25% | ~98% | ~95% of MQA |
| **MQA** | 1 | 1 × d × seq_len | 1/H× = 1.6% | ~95-98% | Baseline (fastest) |
**Converting MHA Checkpoints to GQA**
One key advantage: existing MHA models can be converted to GQA by mean-pooling the KV heads within each group and continuing training (uptraining). This avoids training from scratch.
```
# Convert 64 KV heads → 8 groups
# Each group = mean of 8 consecutive KV heads
group_1_K = mean(K_1, K_2, ..., K_8)
group_2_K = mean(K_9, K_10, ..., K_16)
...
# Then uptrain for ~5% of original training tokens
```
**Models Using GQA**
| Model | Query Heads | KV Heads (Groups) | Ratio |
|-------|------------|-------------------|-------|
| **Llama-2 70B** | 64 | 8 | 8:1 |
| **Mistral 7B** | 32 | 8 | 4:1 |
| **Gemma** | 16 | 1-8 (varies by size) | Varies |
| **Llama-3 8B** | 32 | 8 | 4:1 |
| **Llama-3 70B** | 64 | 8 | 8:1 |
| **Qwen-2** | 28 | 4 | 7:1 |
**Grouped-Query Attention is the recommended default attention architecture for modern LLMs** — providing a configurable KV-cache reduction (4-8× typical) that preserves near-full MHA quality while approaching MQA inference speeds, with the additional advantage of being convertible from existing MHA checkpoints through mean-pooling and uptraining rather than requiring training from scratch.