LLM Quantization & Low-Precision Simulator

Model what dropping an LLM to 8-bit or 4-bit actually buys — and costs — then run it: the simulation executes on the ChipFoundryServices compute pool. Lowering the precision of weights, activations and the KV-cache shrinks the footprint (bytes fall linearly with bit-width, plus a per-group scale overhead of 16 / group_size effective bits), and because decode is memory-bound, fewer bytes read per token means more tokens/s. Compute (matmul) only speeds up when both MAC operands are narrow — FP8/INT8 ≈ 2×, INT4 ≈ 4× the FP16 rate — so weight-only schemes like W4A16 win on bandwidth, not on math. The cost is numerical noise: uniform quantization gives a signal-to-quantization-noise ratio SQNR ≈ 6.02·bits + 1.76 − 20·log₁₀(crest), where a coarse group size (per-tensor) and activation/KV outliers widen the range each scale must cover and erode accuracy. The node returns the compression ratio, decode speed-up, per-source SQNR, the dominant error source and a quality-retention estimate — the same W-A-KV bit-width and grouping trade-offs that decide whether post-training quantization is near-lossless or needs QAT. Reduced-order educational model. See also the KV-cache, HBM bandwidth, 3D-parallelism, training-memory, power & thermal, systolic array, transistor I-V, thermal, interconnect RC, die-yield, 6T SRAM, CMP planarization and lithography simulators and the compute-pool status.

INT8 · W8A8 INT4 weight-only · W4A16 W4A4 · aggressive INT4 · per-tensor
Quality retention vs weight precision — a uniform-quantization SQNR proxy. Accuracy holds near 100 % down to 8-bit, then falls off a knee below ~4-bit as quantization noise swamps the signal. The dashed line is full precision, the amber line the ~90 % usable floor, and the marker (●) is your current setting
Left: decode throughput vs weight precision — decode is memory-bound, so fewer bits per weight means fewer bytes read per token and more tokens/s. Right: the memory footprint — quantized weights and KV-cache as a share of the FP16 total; ◂ marks the larger term
Developer API — same simulation over HTTP (load-balanced across the pool): 
curl -X POST https://www.chipfoundryservices.com/edge/quantization \
  -H "Content-Type: application/json" \
  -d '{"model_params_b":70,"weight_bits":4,"activation_bits":16,"kv_bits":16,
       "group_size":128,"batch_size":32,"seq_length":4096,"hidden_dim":8192,
       "num_layers":80,"hbm_bandwidth_gb_s":3350,"compute_tops_fp16":990}'
Returns JSON with outputs (format, effective_weight_bits, scale_overhead_percent, weight_gb, kv_cache_gb, total_gb, fp16_total_gb, memory_reduction_x, decode_tokens_s, fp16_decode_tokens_s, decode_speedup_x, compute_speedup_x, decode_bottleneck, sqnr_weight_db, sqnr_activation_db, sqnr_kv_db, min_sqnr_db, quant_error_percent, accuracy_retention_percent, error_source, verdict), the full profile (48-point acc_vs_bits accuracy-vs-precision and tps_vs_bits decode-throughput sweeps, plus weight_gb, kv_gb, total_gb, fp16_total_gb, retention_percent, decode_tokens_s, weight_bits, group_size), the serving node, and compute_ms. Endpoint aliases /edge/quant, /edge/precision, /edge/int8, /edge/int4, /edge/fp8, /edge/lowbit.