pinned memory cuda
**Pinned (Page-Locked) Memory** is **host memory that is locked in physical RAM and cannot be swapped to disk** — enabling the GPU to access host memory directly via DMA without CPU involvement and allowing asynchronous (overlapping) memory transfers.
**Why Pinned Memory?**
- Regular (pageable) memory: CPU can swap pages to disk. DMA transfer requires:
1. Allocate temporary pinned buffer.
2. Copy from pageable → pinned (CPU).
3. DMA transfer pinned → GPU.
- Double copy, synchronous.
- Pinned memory: Skip step 1-2 → DMA directly from host.
- 1.5–2x faster transfer bandwidth.
- Enables `cudaMemcpyAsync` — true asynchronous transfer.
**Allocating Pinned Memory**
```cuda
float* h_data;
cudaMallocHost(&h_data, size); // Pinned allocation
cudaFreeHost(h_data); // Free pinned memory
// Async transfer (non-blocking)
cudaMemcpyAsync(d_data, h_data, size, cudaMemcpyHostToDevice, stream);
```
**Zero-Copy Memory**
- Map pinned host memory into GPU address space.
- GPU accesses host memory directly via PCIe (no explicit transfer).
- `cudaHostAlloc(ptr, size, cudaHostAllocMapped)`
- Useful when: Data accessed once (transfer + use = same latency as zero-copy), or host memory larger than GPU memory.
- Slower than transfer + compute: PCIe bandwidth ~16 GB/s vs. GPU memory ~900 GB/s.
**When to Use Pinned Memory**
- Always: For streaming/pipelined workloads with `cudaMemcpyAsync`.
- Large transfers: Bandwidth gain justifies pinning overhead.
- High-frequency small transfers: Saves per-transfer staging cost.
**When NOT to Overuse**
- Pinned memory cannot be swapped → reduces available virtual memory.
- Over-allocation: System runs low on physical memory → performance degradation.
- Rule: Pin only the buffers actively used for DMA transfers.
Pinned memory is **a prerequisite for achieving peak PCIe bandwidth and enabling the transfer-compute overlap** that allows GPU inference and training pipelines to saturate GPU compute without waiting for data transfers.