Mixture-of-Experts Routing Simulator

Model why sparse Mixture-of-Experts is how frontier LLMs scale parameters without scaling compute — then run it: the simulation executes on the ChipFoundryServices compute pool. A dense Transformer runs every token through one big feed-forward network; a MoE layer replaces it with N experts and a gating network that routes each token to only its top-k, so just k / N of the parameters are active and the layer does about N / (capacity·k) less matmul than a dense FFN of the same total size — the whole reason MoE exists. But experts run as fixed-size batches: each accepts capacity = capacity_factor · tokens·k / N tokens, so anything beyond that is dropped (its output zeroed) while cold experts leave slots padded with dead compute. Real routing is never balanced, so the hottest experts overflow and others starve — raising capacity_factor cuts drops but wastes more compute on padding (the classic MoE U-curve). With expert parallelism the experts shard across devices, so every micro-batch does two all-to-all shuffles (dispatch the tokens, combine the results) — traffic that often becomes the real bottleneck at scale. The node returns the sparsity, per-expert load, the drop and padding fractions, the compute saved vs dense and the compute-vs-network bottleneck. Reduced-order educational model. See also the FlashAttention, quantization, KV-cache, HBM bandwidth, systolic array, 3D-parallelism, training-memory, power & thermal, transistor I-V, thermal, die-yield and lithography simulators and the compute-pool status.

Mixtral · 8×top-2 Many experts · 128 All-to-all bound Token dropping
Per-expert load (tokens routed to each expert, sorted hottest-first, normalized so a balanced load = 1.0). The dashed line is the capacity ceiling: the red caps above it are dropped tokens (overflow), the faint gaps below it are padding (slots the accelerator pads with dead compute). Wider spread = higher load imbalance
Left: the MoE capacity U-curve — token drop % falls and padding waste % rises as you raise capacity_factor; the sweet spot is where their sum is smallest (▯ marks your setting). Right: the operating budget — expert-compute vs all-to-all share of the step, slot utilization and active-vs-total parameter sparsity
Developer API — same simulation over HTTP (load-balanced across the pool): 
curl -X POST https://www.chipfoundryservices.com/edge/moe \
  -H "Content-Type: application/json" \
  -d '{"batch_size":16,"seq_length":2048,"num_experts":64,"top_k":2,
       "d_model":4096,"d_ff":14336,"capacity_factor":1.25,"load_imbalance":0.3,
       "expert_parallel":8,"elem_bytes":2,"network_bw_gb_s":450,"compute_tops":990}'
Returns JSON with outputs (format, tokens, routed_assignments, mean_load_per_expert, capacity_per_expert, dropped_tokens, drop_percent, padding_waste_percent, slot_utilization_percent, total_expert_params_b, active_params_b, sparsity_percent, expert_weights_gb, moe_tflop, dense_equiv_tflop, compute_savings_x, all2all_gb, compute_time_ms, all2all_time_ms, step_time_ms, comm_fraction_percent, bottleneck, achieved_tflops, mfu_percent, verdict), the full profile (48-point drop_vs_capacity and waste_vs_capacity sweeps, the per-expert expert_loads array, plus capacity_norm, num_experts, top_k, sparsity_percent, compute_savings_x, drop_percent, padding_waste_percent, comm_fraction_percent, capacity_factor), the serving node, and compute_ms. Endpoint aliases /edge/mixture, /edge/experts, /edge/router, /edge/gating, /edge/sparse.