TDP (Thermal Design Power) is the maximum amount of heat a processor generates under sustained workload — measured in watts, this specification determines cooling requirements and power delivery, directly impacting system design for AI workloads where GPU TDP ranges from 75W to 700W.
What Is TDP?
- Definition: Maximum heat output under sustained load, in watts.
- Purpose: Specifies cooling system requirements.
- Measurement: Sustained power, not peak.
- Relation: Roughly equals power consumption under load.
Why TDP Matters for AI
- Cooling Design: Higher TDP needs larger/better coolers.
- Power Delivery: PSU must supply TDP + headroom.
- Data Center: Determines rack density and cooling capacity.
- Operating Costs: Higher TDP = higher electricity bills.
- Thermal Throttling: Inadequate cooling reduces performance.
GPU TDP Comparison
AI/ML GPUs:
``
GPU | TDP (W) | Memory | Use Case
-----------------|---------|-----------|------------------
NVIDIA H100 SXM | 700 | 80GB HBM3 | Training/Inference
NVIDIA H100 PCIe | 350 | 80GB HBM3 | Inference, lower power
NVIDIA A100 SXM | 400 | 80GB HBM2e| Training/Inference
NVIDIA A100 PCIe | 300 | 80GB HBM2e| Inference
NVIDIA L40S | 350 | 48GB GDDR6| Inference
NVIDIA L4 | 72 | 24GB GDDR6| Edge inference
AMD MI300X | 750 | 192GB HBM3| Training
Consumer GPUs:
`
GPU | TDP (W) | Memory | AI Use
-----------------|---------|-----------|------------------
RTX 4090 | 450 | 24GB | Dev, small training
RTX 4080 Super | 320 | 16GB | Development
RTX 4070 | 200 | 12GB | Inference
RTX 3090 | 350 | 24GB | Budget training
TDP vs. Power Consumption
Understanding the Relationship:
`
TDP: Design thermal envelope (sustained)
Peak Power: Can exceed TDP briefly
Idle Power: Much lower than TDP
Actual Power: Depends on workload
Example (RTX 4090):
TDP: 450W
Peak: ~600W (transient)
Typical gaming: 300-400W
Idle: 20-30W
LLM inference: 250-350W
`
Power Modes:
`
Mode | Power | Performance
---------------|----------|-------------
Full TDP | 100% | 100%
Power limited | 70-80% | 95%
Eco mode | 50-60% | 80%
Undervolted | 80-90% | 100%
Cooling Requirements
Cooling Solutions by TDP:
`
TDP Range | Cooling Type | Noise
-------------|----------------------|-------
<100W | Single fan | Low
100-200W | Dual fan | Medium
200-350W | Triple fan/AIO | Medium-High
350-500W | Custom loop/blower | High
500W+ | Liquid (rack/water) | Varies
Data Center Cooling:
`
Cooling Type | Capacity | Density
-----------------|-------------|-------------------
Air cooling | <30kW/rack | Standard
Rear-door heat | 30-50kW/rack| Medium density
Direct liquid | 50-100kW/rack| High density H100
Immersion | 100kW+/rack | Extreme density
Power Budget Planning
System Power Calculation:
`
Component | Power (W)
-----------------|----------
GPU (H100 SXM) | 700
CPU | 200-350
Memory | 50-100
Storage | 25-50
Networking | 25-50
Misc | 50-100
System total | ~1100-1350W
PSU requirement: 1.5× total = 1650-2000W
`
Rack Planning:
`
8× H100 SXM system: ~10kW
Per-rack capacity: 30-100kW depending on cooling
H100 systems per rack: 3-10
Data center power: MW to hundreds of MW
Efficiency Considerations
Performance per Watt:
`
GPU | TDP | FP16 TFLOPS | TFLOPS/W
------------|------|-------------|----------
H100 SXM | 700W | 1979 | 2.83
H100 PCIe | 350W | 1513 | 4.32
A100 SXM | 400W | 312 | 0.78
L4 | 72W | 121 | 1.68
Optimization:
`
- Power limiting (90% power → 98% perf typical)
- Undervolting for efficiency
- Workload-appropriate GPU selection
- Batch scheduling to maximize utilization
TDP specification is fundamental to AI infrastructure planning — understanding thermal requirements determines cooling design, power delivery, operating costs, and ultimately the density and efficiency of AI compute deployments.