simd vectorization avx

**SIMD Vectorization** is **the CPU optimization technique that processes multiple data elements simultaneously using wide vector registers and instructions — achieving 4-16× throughput improvement for data-parallel operations by executing the same operation on 128-512 bit vectors containing multiple integers or floating-point values in a single clock cycle**. **SIMD Instruction Set Evolution:** - **SSE/SSE2 (128-bit)**: four single-precision floats or two double-precision per instruction — baseline SIMD on x86; all modern x86 CPUs support SSE2 - **AVX/AVX2 (256-bit)**: eight single-precision or four double-precision per instruction — FMA (fused multiply-add) added in AVX2, doubling peak throughput for multiply-accumulate patterns - **AVX-512 (512-bit)**: sixteen single-precision or eight double-precision per instruction — includes mask registers for predicated execution and advanced gather/scatter instructions; available on Intel Xeon and recent consumer processors - **ARM NEON/SVE**: NEON provides 128-bit SIMD on all ARM64 cores; SVE (Scalable Vector Extension) supports variable-length vectors (128-2048 bits) — SVE code runs on any implementation without recompilation **Auto-Vectorization:** - **Compiler Analysis**: modern compilers (GCC -O3, Clang -O3, ICC) analyze loops to identify vectorizable patterns — loop iterations must be independent (no loop-carried dependencies) for vectorization - **Pragmas and Hints**: #pragma omp simd, __attribute__((vectorize)), and restrict keyword help the compiler prove independence — -ffast-math relaxes floating-point semantics to enable more aggressive vectorization - **Vectorization Reports**: -fopt-info-vec (GCC), -Rpass=loop-vectorize (Clang) report which loops were vectorized and why others failed — common failure reasons: aliasing, non-contiguous access, function calls, complex control flow - **SLP Vectorization**: Superword Level Parallelism vectorizes independent scalar operations (not just loops) — packs multiple independent operations on different variables into single vector instruction **Performance Considerations:** - **Alignment**: memory addresses aligned to vector width (16/32/64 bytes) enable aligned loads/stores — unaligned access costs 0-3 extra cycles on modern CPUs; _mm_malloc or alignas() ensure alignment - **Gather/Scatter**: non-contiguous access patterns require gather instructions (vpgatherdd) — 4-8× slower than contiguous vector loads; data layout transformation (AoS→SoA) eliminates gathers - **Masking**: AVX-512 mask registers enable predicated vector operations — partial vector utilization (processing 5 of 16 elements) wastes SIMD lanes but avoids scalar fallback code - **Frequency Throttling**: AVX-512 heavy workloads may trigger CPU frequency reduction (Intel thermal/power management) — net performance gain depends on workload's instruction mix; pure AVX-512 at reduced frequency may not beat AVX2 at full frequency **SIMD vectorization is the most immediately impactful single-core optimization technique — understanding vectorization enables developers to achieve 4-16× speedup on data-parallel code without additional hardware, making it the essential complement to multi-threading for performance-critical applications.**