helm
**Helm Charts for ML Deployments**
**What is Helm?**
Package manager for Kubernetes, using charts (templates) to deploy applications with configurable values.
**Basic Helm Chart Structure**
```svg
```
**Chart.yaml**
```yaml
apiVersion: v2
name: llm-inference
description: LLM inference server
version: 1.0.0
appVersion: "1.0.0"
```
**values.yaml**
```yaml
replicaCount: 2
image:
repository: llm-inference
tag: "v1.0.0"
pullPolicy: IfNotPresent
model:
name: "gpt-4"
maxTokens: 4096
resources:
limits:
nvidia.com/gpu: 1
memory: 16Gi
requests:
nvidia.com/gpu: 1
memory: 8Gi
autoscaling:
enabled: true
minReplicas: 2
maxReplicas: 10
targetGPUUtilization: 70
```
**Deployment Template**
```yaml
# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-llm
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
app: {{ .Release.Name }}
template:
spec:
containers:
- name: llm-server
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
resources:
{{- toYaml .Values.resources | nindent 10 }}
env:
- name: MODEL_NAME
value: {{ .Values.model.name }}
- name: MAX_TOKENS
value: "{{ .Values.model.maxTokens }}"
```
**Install and Upgrade**
```bash
# Install
helm install llm-prod ./llm-inference -f values-prod.yaml
# Upgrade
helm upgrade llm-prod ./llm-inference -f values-prod.yaml
# Rollback
helm rollback llm-prod 1
# Uninstall
helm uninstall llm-prod
```
**Popular ML Helm Charts**
| Chart | Purpose |
|-------|---------|
| vLLM | High-throughput inference |
| text-generation-inference | HuggingFace TGI |
| ray-cluster | Distributed training |
| mlflow | Experiment tracking |
| triton-inference-server | NVIDIA serving |
**Best Practices**
- Use values files per environment
- Version your charts
- Test templates with helm template
- Use helm secrets for sensitive values
- Keep charts in git with application code