Expert Capacity is the maximum number of tokens that can be routed to any single expert within a Mixture-of-Experts (MoE) layer during a single forward pass, defined as the capacity factor (CF) multiplied by the average number of tokens per expert (total tokens / number of experts). Expert capacity acts as a hard buffer limit that prevents memory overflow and ensures balanced computation, but tokens exceeding an expert's capacity are dropped and passed through residual connections without expert processing.

Why Expert Capacity Matters in AI/ML: Expert capacity is a critical design parameter that balances computational efficiency, memory usage, and model quality in MoE architectures—too low causes excessive token dropping, too high wastes memory and computation.

• Capacity factor tuning — CF = 1.0 means each expert has exactly enough buffer for perfectly balanced routing; practical values range from 1.0-1.5 to accommodate routing imbalance; Switch Transformer uses CF = 1.0-1.25 with auxiliary load balancing • Token dropping — When more tokens are routed to an expert than its capacity allows, overflow tokens skip expert processing and pass through the residual connection, degrading quality proportional to the drop rate; well-tuned models target <1% token dropping • Memory planning — Expert capacity directly determines the memory allocated per expert for activation storage during the forward pass; capacity × hidden_dim × batch determines the expert buffer size in GPU memory • Batch size interaction — Larger batch sizes provide better statistical averaging of routing decisions, reducing per-expert load variance and allowing lower capacity factors; small batches require higher CF to avoid excessive dropping • Dynamic capacity — Advanced implementations (e.g., Megablocks, FlexMoE) use variable-length expert buffers to eliminate fixed capacity constraints, processing exactly the tokens routed to each expert without dropping or waste

Expert capacity is the key parameter governing the efficiency-quality tradeoff in MoE architectures, determining how many tokens each expert can process per batch and directly controlling the token-dropping rate that impacts model quality, memory consumption, and computational efficiency of sparse expert models.