cuda programming basics
**CUDA Programming** — NVIDIA's parallel computing platform that enables general-purpose computation on GPUs, leveraging thousands of cores for massive parallelism.
**Execution Hierarchy**
- **Thread**: Smallest unit of execution
- **Warp**: 32 threads executing in lockstep (SIMT)
- **Block**: Group of threads (up to 1024) sharing fast shared memory
- **Grid**: Collection of all blocks launched for a kernel
**Basic Pattern**
```
__global__ void add(float *a, float *b, float *c, int n) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < n) c[i] = a[i] + b[i];
}
// Launch: add<<>>(a, b, c, n);
```
**Memory Hierarchy**
- **Registers**: Fastest, per-thread (~256KB per SM)
- **Shared Memory**: Per-block, programmer-managed cache (~100KB per SM). 100x faster than global
- **Global Memory**: Large (up to 80GB HBM) but slow (~500 cycles latency). Coalesced access is key
**Performance Keys**
- Maximize occupancy (fill SMs with enough warps to hide latency)
- Coalesce global memory accesses (consecutive threads access consecutive addresses)
- Use shared memory to reduce global memory traffic
- Minimize warp divergence (avoid branches within a warp)
**CUDA** powers the majority of GPU computing workloads from deep learning training to scientific simulation.