Speculative Decoding Simulator

Model why a small draft model plus a big target model decodes LLM tokens several times faster — then run it: the simulation executes on the ChipFoundryServices compute pool. Plain autoregressive decoding is memory-bound — emitting one token streams every weight of the big target model from HBM while the matmul barely touches the compute units. Speculative decoding lets a cheap draft model guess the next gamma tokens, then the target verifies all gamma+1 positions in a single forward pass — and because that pass still reads the weights only once, it costs about the same as decoding one token yet can accept several at once. Each drafted token is kept only if the target agrees, so with per-token acceptance alpha the expected tokens confirmed per step is E = (1 - alphagamma+1) / (1 - alpha), saturating at 1/(1-alpha). Three forces set the payoff: the acceptance rate, the draft cost ratio (a draft too big eats the win), and the verify compute crossover — a large gamma pushes the target pass out of the memory-bound regime, capping useful speculation. The node returns the expected tokens/step, the draft/verify latency split, the net speedup over plain decoding, the optimal gamma and the operating regime. Reduced-order educational model. See also the Mixture-of-Experts, FlashAttention, quantization, KV-cache, HBM bandwidth, systolic array, 3D-parallelism, training-memory, power & thermal, transistor I-V, thermal, die-yield and lithography simulators and the compute-pool status.

Llama 1B → 70B High-acceptance draft Draft too big Low acceptance
Speedup vs speculation length γ at your acceptance rate. Each bar is the net throughput gain over plain decoding: it rises as more tokens are accepted per step, then flattens or falls once the draft's sequential cost and the target's verify-compute add up. The dashed line marks 1× (break-even) and ▯ marks your γ; the tallest bar is the optimal γ
Left: speedup vs acceptance rate α — the payoff climbs steeply as the draft aligns with the target (▯ marks your α). Right: the step budget — how the per-step latency splits between the γ sequential draft passes and the single verify pass, plus expected-vs-max tokens (block efficiency) and the draft cost ratio
Developer API — same simulation over HTTP (load-balanced across the pool): 
curl -X POST https://www.chipfoundryservices.com/edge/specdec \
  -H "Content-Type: application/json" \
  -d '{"gamma":4,"alpha":0.8,"draft_params_b":1,"target_params_b":70,
       "elem_bytes":2,"hbm_bw_gb_s":3350,"compute_tops":990,"batch_size":1}'
Returns JSON with outputs (format, expected_tokens_per_step, max_tokens_per_step, block_efficiency_percent, acceptance_percent, speedup_x, baseline_tokens_s, spec_tokens_s, target_decode_ms, draft_step_ms, draft_total_ms, verify_step_ms, spec_step_ms, draft_cost_ratio, draft_overhead_percent, verify_fraction_percent, verify_bottleneck, verify_mem_ms, verify_compute_ms, target_bound, optimal_gamma, optimal_speedup_x, verdict), the full profile (16-point speedup_vs_gamma and tokens_vs_gamma sweeps, 48-point speedup_vs_alpha sweep, plus gamma, alpha, optimal_gamma, speedup_x, expected_tokens_per_step, max_tokens_per_step, draft_overhead_percent, verify_fraction_percent, draft_cost_ratio), the serving node, and compute_ms. Endpoint aliases /edge/speculative, /edge/draft, /edge/assisted, /edge/lookahead, /edge/medusa.