Truss: Model Packaging & Deployment

Overview Truss is an open-source framework (by Baseten) for packaging AI/ML models. It solves the "it works on my machine" problem for ML models by creating a standardized structure that runs locally and deploys anywhere (Docker).

The Problem Deploying a model requires:

• Correct Python version.
• System packages (apt-get install libGL).
• Python requirements (pip install torch).
• Serialization checks.

Truss handles this automatically.

How to use

pip install truss

import truss
from transformers import pipeline

# Load model
pipe = pipeline("text-classification")

# Create truss
truss.create(pipe, target_directory="./my-model")

This creates a folder with:

• model/model.py: Inference logic.
• config.yaml: Dependencies and settings.
• data/: Model weights.

Live Reload Truss supports "live reload" during development. You can tweak the model.py code and verify the API response instantly in Docker without rebuilding the image from scratch.

Deployment

• Baseten: Native deployment (one click).
• Docker: truss build-image ./my-model → deploy to AWS/GCP.

Truss is a modern alternative to BentoML, focusing on developer experience and rapid iteration.