vast.ai

**Vast.ai** is the **peer-to-peer GPU marketplace enabling ML practitioners to rent consumer and data center GPUs from individual hosts at 4-10x lower cost than cloud providers** — trading guaranteed reliability for extreme cost efficiency through a marketplace model where GPU owners list their hardware and researchers bid for compute time via Docker containers. **What Is Vast.ai?** - **Definition**: A decentralized GPU marketplace founded in 2017 where GPU owners (sellers) list their hardware and ML practitioners (buyers) rent compute via Docker containers — with pricing determined by supply and demand rather than fixed cloud provider rates. - **Peer-to-Peer Model**: Sellers install the Vast.ai client on their machines (gaming PCs, mining farms, colocation servers), connecting their GPUs to the marketplace. Buyers browse instances filtered by GPU type, price, location, and reliability score. - **Docker-Based**: All rentals run as Docker containers — buyers specify their Docker image (e.g., pytorch/pytorch:2.0-cuda11.7) and the host machine runs it with full root access inside the container. - **Pricing**: Market-driven — RTX 4090s available at $0.30-0.50/hr, A100s at $0.80-1.20/hr, H100s at $1.50-2.00/hr. Interruptible instances offer further discounts at the cost of potential termination. - **Reliability Spectrum**: Reliability scores (0-100) indicate host uptime history — score 99+ indicates data center hardware; score 70-80 indicates a gaming PC that may go offline unexpectedly. **Why Vast.ai Matters for AI** - **Extreme Cost Reduction**: 4-10x cheaper than AWS/GCP for equivalent GPU — a week of A100 training that costs $3,000 on AWS costs $600-800 on Vast.ai, making research accessible on limited budgets. - **RTX 4090 Access**: Consumer RTX 4090s (24GB VRAM) available at $0.30-0.50/hr — this GPU type is unavailable on AWS/GCP but excellent for fine-tuning models up to 13B parameters with quantization. - **No Commitment**: Rent by the hour, no minimum contract, no reserved instance commitment — ideal for experiments, one-off training runs, and model evaluation. - **Budget Research**: Students, independent researchers, and early-stage startups use Vast.ai to access GPU hardware that would otherwise require enterprise cloud budgets. - **Spot-Like Pricing**: When market demand is low, compute available below listed prices through bidding — aggressive bids can get 30-50% discounts on available instances. **Vast.ai Key Concepts** **Instance Types**: - **On-Demand**: Pay listed hourly price, instance runs until manually stopped - **Interruptible**: Bid below listed price, instance runs until host reclaims GPU — cheaper but can terminate mid-run - **Reserved**: Longer-term rental at negotiated price with stability commitment **Reliability Scores**: - Vast.ai tracks host uptime, internet bandwidth, and interrupt frequency over time - Filter by reliability score when stability matters: choose 95+ for multi-day runs - Lower scores acceptable for short experiments where interruption is tolerable **Docker Workflow**: 1. Browse marketplace, filter by GPU type and price 2. Select instance and specify Docker image 3. Launch — SSH access available in 1-5 minutes 4. Run training, save checkpoints to persistent storage or S3 5. Terminate instance — pay only for active hours **Good Fit vs Poor Fit** **Good for Vast.ai**: - One-off fine-tuning runs (2-12 hours) - Hyperparameter search experiments - Model evaluation and benchmarking - Learning and experimentation on limited budget - RTX 4090 access for medium-scale fine-tuning **Avoid for Vast.ai**: - Production inference serving requiring uptime SLAs - Long multi-week training runs with interruption risk - Regulated workloads (HIPAA, SOC2 compliance unavailable) - Multi-node distributed training requiring reliable networking **Vast.ai vs Alternatives** | Provider | Cost | Reliability | GPU Types | Best For | |----------|------|------------|-----------|---------| | Vast.ai | Lowest | Low-Medium | Consumer + DC | Budget experiments | | RunPod Community | Low | Medium | Consumer + DC | Budget training | | Lambda Labs | Low-Medium | High | DC (H100, A100) | Reliable ML training | | CoreWeave | Medium | Very High | DC only | Enterprise scale | | AWS/GCP | High | Very High | DC only | Production, compliance | Vast.ai is **the go-to marketplace for budget-conscious ML practitioners who prioritize compute cost over guaranteed reliability** — by connecting GPU owners directly with renters, Vast.ai makes frontier-class GPUs accessible at hobbyist prices and enables ML research that would otherwise require enterprise cloud budgets.