Helm Charts for ML Deployments

What is Helm? Package manager for Kubernetes, using charts (templates) to deploy applications with configurable values.

Basic Helm Chart Structure

<svg viewBox="0 0 233 188" xmlns="http://www.w3.org/2000/svg" style="max-width:100%;height:auto" role="img"><rect x="0" y="0" width="233" height="188" rx="12" fill="#0d1117"/><g font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,&quot;Liberation Mono&quot;,monospace" font-size="14"><text xml:space="preserve" x="20" y="31.7"><tspan fill="#c9d1d9">llm-inference/</tspan></text><text xml:space="preserve" x="20" y="50.7"><tspan fill="#6e7681">├──</tspan><tspan fill="#c9d1d9"> Chart.yaml</tspan></text><text xml:space="preserve" x="20" y="69.7"><tspan fill="#6e7681">├──</tspan><tspan fill="#c9d1d9"> values.yaml</tspan></text><text xml:space="preserve" x="20" y="88.7"><tspan fill="#6e7681">├──</tspan><tspan fill="#c9d1d9"> templates/</tspan></text><text xml:space="preserve" x="20" y="107.7"><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">   </tspan><tspan fill="#6e7681">├──</tspan><tspan fill="#c9d1d9"> deployment.yaml</tspan></text><text xml:space="preserve" x="20" y="126.7"><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">   </tspan><tspan fill="#6e7681">├──</tspan><tspan fill="#c9d1d9"> service.yaml</tspan></text><text xml:space="preserve" x="20" y="145.7"><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">   </tspan><tspan fill="#6e7681">├──</tspan><tspan fill="#c9d1d9"> configmap.yaml</tspan></text><text xml:space="preserve" x="20" y="164.7"><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">   </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9"> hpa.yaml</tspan></text></g></svg>

Chart.yaml

apiVersion: v2
name: llm-inference
description: LLM inference server
version: 1.0.0
appVersion: "1.0.0"

values.yaml

replicaCount: 2

image:
  repository: llm-inference
  tag: "v1.0.0"
  pullPolicy: IfNotPresent

model:
  name: "gpt-4"
  maxTokens: 4096

resources:
  limits:
    nvidia.com/gpu: 1
    memory: 16Gi
  requests:
    nvidia.com/gpu: 1
    memory: 8Gi

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 10
  targetGPUUtilization: 70

Deployment Template

# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Release.Name }}-llm
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Release.Name }}
  template:
    spec:
      containers:
      - name: llm-server
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
        resources:
          {{- toYaml .Values.resources | nindent 10 }}
        env:
        - name: MODEL_NAME
          value: {{ .Values.model.name }}
        - name: MAX_TOKENS
          value: "{{ .Values.model.maxTokens }}"

Install and Upgrade

# Install
helm install llm-prod ./llm-inference -f values-prod.yaml

# Upgrade
helm upgrade llm-prod ./llm-inference -f values-prod.yaml

# Rollback
helm rollback llm-prod 1

# Uninstall
helm uninstall llm-prod

Popular ML Helm Charts

Best Practices

• Use values files per environment
• Version your charts
• Test templates with helm template
• Use helm secrets for sensitive values
• Keep charts in git with application code