Batch size optimization tunes the number of concurrent requests processed together during LLM inference to maximize throughput while meeting latency requirements, balancing GPU utilization against response time. Key tradeoff: (1) Small batch—low latency per request but underutilizes GPU compute (especially during decode phase); (2) Large batch—high throughput and GPU utilization but increased per-request latency and memory pressure. Batch size constraints: (1) GPU memory—each request requires KV cache storage (grows with sequence length), limiting maximum batch size; (2) Latency SLO—maximum acceptable time-to-first-token and inter-token delay; (3) Compute saturation—point where adding more requests doesn't increase tokens/second. Memory calculation: KV cache per request = 2 × n_layers × n_heads × head_dim × seq_len × dtype_bytes. For 70B model with 4K context in FP16: ~10GB per request, limiting H100 (80GB) to ~4-5 concurrent requests (after model weights). Optimization strategies: (1) Quantized KV cache—INT8 or FP8 cache doubles batch capacity; (2) Multi-query attention (MQA)/grouped-query attention (GQA)—reduces KV cache size 8-32×; (3) PagedAttention—eliminates memory fragmentation, maximizes usable memory; (4) Dynamic batching—adjust batch size based on current load and request characteristics; (5) Prefix caching—share KV cache for common prompt prefixes. Profiling approach: sweep batch sizes, measure throughput (tokens/s) and latency (P50/P99), find knee of curve where throughput plateaus before latency degrades. Different batch sizes for prefill vs. decode: chunked prefill processes long inputs in smaller chunks to avoid blocking decode of other requests. Optimal batch size is workload-dependent—varies with model size, sequence length distribution, hardware, and latency requirements.