Home Knowledge Base Process

Post-Training Quantization (PTQ) compresses trained models to lower precision without retraining. Process: Take trained FP32/FP16 model → analyze weight and activation distributions → determine quantization parameters (scale, zero-point) → convert to INT8/INT4 → calibrate with representative data. Quantization types: Weight-only (easier, good for memory-bound), weight-and-activation (better speedup, needs calibration), static (fixed ranges), dynamic (runtime computation). Calibration: Run representative dataset through model, collect activation statistics (min/max, percentiles), set quantization ranges to minimize error. Per-tensor vs per-channel: Per-channel captures weight variation better, especially for convolutions and linear layers with diverse distributions. Tools: PyTorch quantization, TensorRT, ONNX Runtime, llama.cpp, GPTQ, AWQ. Quality considerations: Sensitive layers may need higher precision, outliers cause accuracy loss, larger models generally more robust to quantization. Results: 2-4x memory reduction, 2-4x inference speedup on supported hardware, typically <1% accuracy loss with INT8, larger degradation at INT4 without careful techniques.

post-training quantization (ptq)post-training quantizationptqmodel optimization

Explore 500+ Semiconductor & AI Topics

From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.