profiler

GPU profilers like NVIDIA Nsight and AMD rocprof identify performance bottlenecks by measuring compute utilization, memory bandwidth, occupancy, and kernel execution metrics—essential tools for optimizing GPU workloads. Nsight Compute: detailed kernel-level analysis (instruction throughput, memory access patterns, warp occupancy), roofline analysis (comparing to theoretical peaks), and bottleneck identification (compute-bound vs memory-bound). Nsight Systems: system-wide profiling (CPU-GPU interactions, CUDA API calls, memory transfers), timeline visualization, and identifying host-device synchronization overhead. AMD rocprof: performance counter collection, kernel timing, and hardware metrics for AMD GPUs. Key metrics to measure: SM/CU occupancy (active warps vs maximum), memory bandwidth utilization (achieved vs peak), arithmetic intensity (compute per byte transferred), and kernel launch overhead. Common bottlenecks: memory-bound (optimize access patterns, use shared memory), compute-bound (algorithm efficiency), latency-bound (small kernels, synchronization), and host-device transfer bound (overlap computation and communication). Optimization workflow: profile → identify bottleneck → optimize → re-profile. Profiling is essential before optimization—intuition about bottlenecks is often wrong. Modern deep learning frameworks integrate with profilers for end-to-end training analysis.