rocm

**ROCm (Radeon Open Compute)** is AMD's **open-source GPU compute platform** designed to provide an alternative to NVIDIA's CUDA for general-purpose GPU computing, including machine learning training and inference. It aims to break NVIDIA's monopoly on GPU-accelerated AI. **ROCm Architecture** - **HIP (Heterogeneous-Compute Interface for Portability)**: The programming interface — syntactically similar to CUDA, making porting relatively straightforward. HIP code can compile for both AMD and NVIDIA GPUs. - **rocBLAS**: Optimized BLAS (Basic Linear Algebra Subprograms) library for AMD GPUs — equivalent to NVIDIA's cuBLAS. - **MIOpen**: Deep learning primitives library — equivalent to NVIDIA's cuDNN. Provides optimized convolutions, attention, and other operations. - **RCCL**: Multi-GPU communication library — equivalent to NVIDIA's NCCL. - **Composable Kernel**: Low-level framework for writing custom GPU kernels. **ROCm for AI/ML** - **PyTorch Support**: ROCm has official PyTorch support — `pip install torch --index-url https://download.pytorch.org/whl/rocm6.0`. - **Training**: AMD Instinct MI250 and MI300X GPUs with ROCm are used for training large models. MI300X has 192GB HBM3 (more than H100's 80GB). - **Inference**: vLLM, TGI, and other inference engines have ROCm support. - **JAX**: ROCm support for JAX is developing but less mature than CUDA support. **AMD GPUs for AI** - **MI300X** (2024): 192GB HBM3, 1.3TB/s memory bandwidth. AMD's flagship AI accelerator. - **MI250** (2022): 128GB HBM2e. Used in the Frontier supercomputer (world's first exascale computer). - **Consumer GPUs**: ROCm support for Radeon RX 7000 series for local model inference. **Challenges** - **Ecosystem Gap**: CUDA has 15+ years of library development, tutorials, and community. ROCm is catching up but the gap is significant. - **Software Maturity**: Some libraries and frameworks have rough edges or incomplete support on ROCm. - **Custom Kernels**: Most custom CUDA kernels (FlashAttention, fused operations) need manual porting to ROCm. - **Enterprise Adoption**: Most AI teams default to NVIDIA — switching requires testing and validation. **Why ROCm Matters** - **Competition**: AMD provides price/performance competition to NVIDIA, preventing monopoly pricing. - **Open Source**: ROCm is fully open-source, unlike CUDA which is proprietary. - **Memory Advantage**: MI300X's 192GB vs H100's 80GB enables running larger models without tensor parallelism. ROCm is the **most credible alternative** to CUDA for large-scale AI workloads, and its momentum is accelerating as AMD invests heavily in AI software.