Candle is a minimalist machine learning framework written in Rust by the Hugging Face team, designed for deploying ML models as small, fast, serverless binaries — eliminating the multi-gigabyte Python/PyTorch dependency chain by compiling models into lightweight Rust executables ideal for AWS Lambda, edge devices, and production environments where low latency, small memory footprint, and startup time matter more than training flexibility.

What Is Candle?

• Definition: A Rust-native tensor computation library with a PyTorch-like API — providing the core operations (matrix multiplication, convolution, attention, normalization) needed to run neural network inference, compiled to native machine code without Python runtime overhead.
• Hugging Face Project: Developed by the Hugging Face team to enable Rust-based model serving — Candle can load model weights from the Hugging Face Hub and run inference using the same pretrained models available to the Python Transformers library.
• Serverless Optimized: Python + PyTorch creates container images of 2-5 GB with 5-10 second cold starts. Candle compiles to binaries under 50 MB with sub-second cold starts — transformative for serverless deployment (AWS Lambda, Cloudflare Workers).
• PyTorch-Like Syntax: Candle's Rust API is designed to feel familiar to PyTorch developers — Tensor::new, tensor.matmul(), tensor.softmax() mirror PyTorch conventions, reducing the learning curve for Rust newcomers.

Key Features

• Model Implementations: Candle includes Rust implementations of Whisper, LLaMA, Mistral, Phi, Stable Diffusion, BERT, and other popular models — ready to use for inference.
• CUDA and Metal Support: GPU acceleration via CUDA (NVIDIA) and Metal (Apple Silicon) — not limited to CPU inference.
• WASM Compilation: Candle models compile to WebAssembly — enabling ML inference in web browsers without a server backend.
• Quantization: Support for GGUF and GGML quantized model loading — run quantized LLMs with Candle's Rust inference engine.
• No Python Dependency: The entire inference stack is pure Rust — no Python interpreter, no pip packages, no virtual environments.

Candle vs Alternatives

Candle is the Rust ML framework that makes model deployment as lightweight as a compiled binary — eliminating Python overhead to deliver sub-second cold starts and tiny container images for serverless and edge deployment, while maintaining access to the Hugging Face model ecosystem through native Rust implementations.