raft
**RAFT** is the **high-accuracy optical flow architecture that uses all-pairs correlation and recurrent iterative updates to refine motion estimates** - instead of coarse-to-fine shrinking of search space, it repeatedly optimizes flow on a fixed high-resolution field.
**What Is RAFT?**
- **Definition**: Recurrent All-Pairs Field Transforms model for dense optical flow.
- **Core Feature**: 4D correlation volume containing pairwise matching scores for all pixel locations.
- **Update Mechanism**: GRU-like recurrent unit performs many refinement iterations.
- **Resolution Strategy**: Maintains a single flow field scale and improves it step by step.
**Why RAFT Matters**
- **State-of-the-Art Accuracy**: Strong benchmark performance on challenging flow datasets.
- **Refinement Stability**: Iterative updates correct errors gradually and robustly.
- **Fine Detail Recovery**: Handles small structures and thin boundaries better than many older methods.
- **Generalization Strength**: Performs well across synthetic and real-world motion domains.
- **System Impact**: Became a dominant flow backbone for many downstream tasks.
**RAFT Architecture**
**All-Pairs Correlation**:
- Precompute dense correspondence tensor between frame feature maps.
- Provide rich search space for iterative updates.
**Recurrent Update Block**:
- Query correlation pyramid and current flow to predict residual update.
- Repeat for fixed number of iterations.
**Context Encoder**:
- Extract static scene context to guide recurrent optimization.
- Improves convergence and boundary precision.
**How It Works**
**Step 1**:
- Encode both frames, build all-pairs correlation volume, initialize flow field.
**Step 2**:
- Run recurrent update loop to refine flow iteratively until convergence.
RAFT is **a refinement-centric optical flow model that turns dense correspondence into precise motion estimates through iterative optimization** - it set a new standard for high-quality flow prediction in modern video pipelines.