LLM Training-Memory Simulator

Model the memory a single LLM training step needs — then run it: the simulation executes on the ChipFoundryServices distributed compute pool. Inference only caches weights and the KV cache; training is far hungrier. A mixed-precision Adam step holds, for every parameter, a low-precision weight and gradient plus an fp32 master weight and Adam’s two moments — the classic 16 bytes per parameter of model states — on top of the activations saved for the backward pass, which scale with batch × sequence × hidden × layers and dwarf the model states at long sequence. From your model geometry and cluster it computes the per-GPU HBM a step needs, how ZeRO / FSDP sharding (stage 1 optimizer, stage 2 + gradients, stage 3 + parameters) drives per-GPU model memory toward params×bytes / GPUs, how activation checkpointing trades compute for memory to make long sequences fit, whether the step fits in HBM or OOMs, and the headline number: the minimum GPU count the run requires. Reduced-order educational model. See also the KV-cache, HBM bandwidth, systolic array, transistor I-V, thermal, interconnect RC, die-yield, 6T SRAM, CMP planarization and lithography simulators and the compute-pool status.

Llama-3-8B · 8×H100 Llama-3-70B · 64×H100 Llama-3-405B · 512 GPU
Per-GPU memory vs GPU count (log–x) — activations (dashed baseline) are fixed per GPU; the sharded model states fall as you add GPUs (ZeRO/FSDP), so per-GPU memory drops toward the activation floor until it slips under the capacity ceiling and the step fits. Your GPU count (●) sits on the curve; the gap down to the roof is your headroom
Left: per-GPU memory vs sequence length — activations grow with sequence, so long context is what makes a step OOM (dashed line = capacity). Right: the per-GPU budget — model states + activations as a share of HBM. Marker is your sequence length
Developer API — same simulation over HTTP (load-balanced across the pool): 
curl -X POST https://www.chipfoundryservices.com/edge/training \
  -H "Content-Type: application/json" \
  -d '{"model_params_b":8,"num_gpus":8,"gpu_memory_gb":80,"micro_batch":4,
       "seq_length":4096,"num_layers":32,"hidden_dim":4096,
       "precision_bits":16,"zero_stage":3,"grad_ckpt":1}'
Returns JSON with outputs (model_states_unsharded_gb, model_states_per_gpu_gb, activations_per_gpu_gb, per_gpu_total_gb, gpu_capacity_gb, utilization_percent, fits, bytes_per_param, weights_per_gpu_gb, grads_per_gpu_gb, optimizer_per_gpu_gb, min_gpus_to_fit, aggregate_cluster_gb, global_batch, zero_stage, checkpointing, verdict), the full profile (48-point mem_vs_gpus and mem_vs_seq sweeps, plus model_state_gb, activation_gb, capacity_gb, per_gpu_gb, num_gpus, seq_length, fits), the serving node, and compute_ms. Endpoint aliases /edge/train, /edge/finetune, /edge/fsdp, /edge/zero.