model compression

Model compression reduces model size and compute requirements through techniques like pruning, quantization, and distillation. **Why compress**: Deployment on edge devices, reduce serving costs, lower latency, fit in memory constraints. **Main techniques**: **Quantization**: Reduce precision (FP32 to INT8, INT4). 2-4x size reduction. **Pruning**: Remove unimportant weights or structures. Variable reduction. **Distillation**: Train small model to mimic large one. Design smaller architecture. **Combined approaches**: Often stack techniques - distill, then quantize and prune. **Accuracy trade-off**: Compression usually reduces accuracy slightly. Goal is minimal degradation for significant efficiency gains. **Structured vs unstructured**: Structured compression (remove whole channels/layers) gives real speedup. Unstructured (sparse weights) needs specialized hardware. **Tools**: TensorRT (NVIDIA), OpenVINO (Intel), ONNX Runtime, Core ML, llama.cpp, GPTQ, AWQ. **LLM compression**: Quantization most impactful (4-bit models common). Pruning and distillation also used. **Evaluation**: Measure accuracy retention, actual speedup, memory reduction. Paper claims vs real deployment may differ.