gpu operator
**GPU Management in Kubernetes**
**NVIDIA GPU Operator**
Automates GPU driver installation, container toolkit, and device plugins in Kubernetes.
**Installation**
```bash
# Add NVIDIA Helm repo
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
# Install GPU Operator
helm install gpu-operator nvidia/gpu-operator
--namespace gpu-operator --create-namespace
```
**Components Installed**
| Component | Purpose |
|-----------|---------|
| Driver | NVIDIA GPU drivers |
| Container Toolkit | nvidia-container-runtime |
| Device Plugin | Expose GPUs to K8s scheduler |
| DCGM Exporter | GPU metrics for Prometheus |
| MIG Manager | Multi-Instance GPU config |
**Requesting GPUs in Pods**
```yaml
apiVersion: v1
kind: Pod
metadata:
name: llm-server
spec:
containers:
- name: inference
image: llm-inference:latest
resources:
limits:
nvidia.com/gpu: 1 # Request 1 GPU
```
**Multiple GPUs**
```yaml
resources:
limits:
nvidia.com/gpu: 4 # Multi-GPU for large models
```
**Node Selectors for GPU Types**
```yaml
spec:
nodeSelector:
nvidia.com/gpu.product: "NVIDIA-A100-SXM4-80GB"
containers:
- name: model
resources:
limits:
nvidia.com/gpu: 1
```
**GPU Sharing (Time-Slicing)**
```yaml
# ConfigMap for time-slicing
apiVersion: v1
kind: ConfigMap
metadata:
name: time-slicing-config
data:
any: |-
version: v1
sharing:
timeSlicing:
resources:
- name: nvidia.com/gpu
replicas: 4 # 4 pods can share each GPU
```
**MIG (Multi-Instance GPU)**
Split A100/H100 into multiple instances:
```yaml
resources:
limits:
nvidia.com/mig-3g.20gb: 1 # 3GB compute, 20GB memory slice
```
**Monitoring GPUs**
```bash
# Check GPU allocation
kubectl describe nodes | grep nvidia.com/gpu
# View GPU metrics
kubectl logs -n gpu-operator dcgm-exporter-xxx
# GPU utilization in Grafana via DCGM metrics
```
**Best Practices**
- Use GPU Operator for consistent setup
- Set appropriate GPU limits
- Use node selectors for GPU types
- Monitor GPU utilization
- Consider time-slicing for dev environments
- Use MIG for flexible resource allocation