iac
**Infrastructure as Code for ML**
**Why IaC for ML?**
Reproducible, version-controlled infrastructure for ML pipelines, training clusters, and inference services.
**Terraform Basics**
```hcl
# Provider configuration
provider "aws" {
region = "us-east-1"
}
# GPU instance for inference
resource "aws_instance" "llm_server" {
ami = "ami-xxx" # Deep Learning AMI
instance_type = "g4dn.xlarge"
tags = {
Name = "llm-inference-server"
}
}
# S3 bucket for models
resource "aws_s3_bucket" "models" {
bucket = "company-llm-models"
}
```
**EKS Cluster for ML**
```hcl
module "eks" {
source = "terraform-aws-modules/eks/aws"
cluster_name = "ml-cluster"
cluster_version = "1.28"
node_groups = {
cpu = {
instance_types = ["m5.2xlarge"]
capacity_type = "ON_DEMAND"
desired_size = 3
}
gpu = {
instance_types = ["g4dn.xlarge"]
capacity_type = "SPOT"
desired_size = 2
labels = {
"nvidia.com/gpu" = "true"
}
taints = [{
key = "nvidia.com/gpu"
value = "true"
effect = "NO_SCHEDULE"
}]
}
}
}
```
**Model Serving Infrastructure**
```hcl
# Load balancer
resource "aws_lb" "llm_api" {
name = "llm-api-lb"
load_balancer_type = "application"
subnets = var.public_subnets
}
# Auto-scaling group
resource "aws_autoscaling_group" "llm_servers" {
desired_capacity = 2
max_size = 10
min_size = 1
launch_template {
id = aws_launch_template.llm_server.id
version = "$Latest"
}
# Scale based on GPU utilization
target_tracking_configuration {
customized_metric_specification {
metric_name = "GPUUtilization"
namespace = "Custom/ML"
statistic = "Average"
}
target_value = 70.0
}
}
```
**Pulumi (Python IaC)**
```python
import pulumi
import pulumi_aws as aws
# GPU instance
llm_server = aws.ec2.Instance("llm-server",
instance_type="g4dn.xlarge",
ami="ami-xxx",
tags={"Name": "llm-inference"}
)
# Export endpoint
pulumi.export("server_ip", llm_server.public_ip)
```
**ML-Specific Resources**
| Resource | Purpose |
|----------|---------|
| GPU instances | Training/inference |
| S3/GCS buckets | Model storage |
| ElastiCache/Redis | Caching |
| SageMaker endpoints | Managed inference |
| Vector databases | RAG storage |
**Best Practices**
- Use modules for reusable components
- Separate environments (dev/staging/prod)
- Store state remotely (S3, Terraform Cloud)
- Use variables for configuration
- Tag resources for cost tracking