Cog is an open-source tool by Replicate that packages machine learning models into standard, production-ready Docker containers — solving the "works on my machine" problem by using a simple cog.yaml configuration file to automatically generate Dockerfiles with correct CUDA drivers, Python versions, system dependencies, and a standardized HTTP prediction API, turning any Python model into a deployable container without writing a single line of Docker configuration.

What Is Cog?

• Definition: A command-line tool (pip install cog) that takes a Python prediction class and a YAML configuration file and produces a fully functional Docker container with an HTTP API at /predictions — handling all the CUDA, system library, and Python dependency complexity automatically.
• The Problem: Data scientists train models in Jupyter notebooks with a chaotic mix of pip, conda, system packages, and specific CUDA versions. Getting this into a Docker container requires deep DevOps knowledge — writing Dockerfiles, managing CUDA driver compatibility, setting up HTTP endpoints, and handling GPU memory.
• The Solution: Define dependencies in cog.yaml, write a predict() function, run cog build — done. Cog generates the Dockerfile, builds the container, and provides a standardized API.

How Cog Works

cog.yaml Example

build:
  gpu: true
  python_version: "3.10"
  python_packages:
    - torch==2.1
    - transformers==4.36
  system_packages:
    - ffmpeg

predict: "predict.py:Predictor"

predict.py Example

from cog import BasePredictor, Input, Path

class Predictor(BasePredictor):
    def setup(self):
        """Load model into memory (runs once on startup)"""
        self.model = load_model("weights/model.pt")

    def predict(self, image: Path = Input(description="Input image")) -> Path:
        """Run inference on an input image"""
        output = self.model(image)
        return Path(output)

Cog vs Alternatives

Cog is the fastest path from ML model to production Docker container — eliminating the DevOps complexity of CUDA drivers, system dependencies, and HTTP API setup through a simple YAML configuration and Python prediction class, enabling data scientists to package any model into a standardized, deployable container without Docker expertise.