zero-copy memory

**Zero-copy memory** is the **memory access model where GPU reads host-resident data directly without explicit copy into device memory** - it removes setup copy overhead but pays higher per-access latency compared with local VRAM. **What Is Zero-copy memory?** - **Definition**: Mapped host memory accessed by GPU through interconnect address translation. - **Benefit**: No explicit memcpy stage before kernel launch for suitable workloads. - **Latency Tradeoff**: Remote host access is slower than accessing local global memory on device. - **Best Fit**: Useful for read-once, low-reuse, or integrated-memory scenarios. **Why Zero-copy memory Matters** - **Simplified Flow**: Can reduce programming overhead for streaming or sparse host-resident inputs. - **Startup Speed**: Avoids copy setup cost for small or transient data payloads. - **Memory Flexibility**: Allows processing of data not fully staged in VRAM. - **Edge Use Cases**: Practical in embedded systems with tighter shared-memory integration. - **Prototype Utility**: Helpful for quick experiments before full transfer optimization. **How It Is Used in Practice** - **Access Profiling**: Measure whether zero-copy latency is acceptable for target kernel pattern. - **Pattern Restriction**: Use zero-copy for low-reuse paths and avoid repeated random access hot loops. - **Hybrid Strategy**: Combine zero-copy for cold paths with explicit copies for high-reuse tensors. Zero-copy memory is **a niche but useful data-access strategy for specific GPU workflows** - it trades lower setup overhead for higher access latency, so workload fit is critical.