h100
**NVIDIA Datacenter GPUs: H100 vs A100**
**NVIDIA H100 (Hopper Architecture)**
The H100 is NVIDIA's flagship AI accelerator, designed specifically for large language models and generative AI workloads.
**H100 Specifications**
| Spec | H100 SXM | H100 PCIe |
|------|----------|-----------|
| Memory | 80GB HBM3 | 80GB HBM3 |
| Bandwidth | 3.35 TB/s | 2.0 TB/s |
| TDP | 700W | 350W |
| Tensor TFLOPs (FP8) | 3,958 | 1,979 |
| NVLink | 900 GB/s | 600 GB/s |
**Key H100 Features**
- **Transformer Engine**: Dynamic FP8/FP16 precision switching
- **2nd Gen MIG**: Up to 7 isolated instances per GPU
- **NVLink 4.0**: 18 links for multi-GPU scaling
**NVIDIA A100 (Ampere Architecture)**
The A100 remains widely deployed and cost-effective for many workloads.
**A100 Specifications**
| Spec | A100 80GB | A100 40GB |
|------|-----------|-----------|
| Memory | 80GB HBM2e | 40GB HBM2e |
| Bandwidth | 2.0 TB/s | 1.6 TB/s |
| TDP | 400W | 400W |
| Tensor TFLOPs (TF32) | 312 | 312 |
**Performance Comparison**
- H100 is approximately **3x faster** than A100 for LLM inference
- For training, H100 offers **2-4x speedup** depending on workload
- A100 still excellent value for many production workloads
**Use Cases**
- **H100**: Large LLM training, real-time inference requiring lowest latency
- **A100**: Cost-effective inference, smaller model training, batch processing