vectorization
**Vectorization** is a **code optimization technique that processes multiple data elements simultaneously using SIMD instructions** — replacing loops with vector operations for 4-16× speedups on numerical computations.
**What Is Vectorization?**
- **Definition**: Apply operations to entire arrays instead of element-by-element.
- **Hardware**: Uses SIMD (Single Instruction Multiple Data) units.
- **Examples**: SSE, AVX (x86), NEON (ARM).
- **Languages**: NumPy, PyTorch, TensorFlow leverage vectorization.
- **Effect**: Process 4-16 elements per instruction.
**Why Vectorization Matters**
- **Speed**: 4-16× faster than scalar loops.
- **NumPy/PyTorch**: Already vectorized — use array operations.
- **Automatic**: Compilers can auto-vectorize simple loops.
- **ML/AI**: All tensor operations are vectorized.
- **Efficiency**: Better CPU/GPU utilization.
**Vectorized vs Loop**
```python
# Slow: Loop
for i in range(len(a)):
c[i] = a[i] + b[i]
# Fast: Vectorized
c = a + b # NumPy
```
**Best Practices**
- Avoid Python loops over arrays.
- Use NumPy/PyTorch operations.
- Batch operations when possible.
- Profile to verify vectorization.
**Auto-Vectorization**
Compilers can vectorize: simple loops, no dependencies, predictable access.
Use: -O3, -march=native for maximum vectorization.
Vectorization is **fundamental for numerical performance** — always prefer array operations over loops.