scene flow estimation
**Scene flow estimation** is the **task of predicting 3D motion vectors for scene points over time, extending optical flow from image-plane displacement to physical-space dynamics** - it combines geometry and motion to model real-world movement in x, y, and z dimensions.
**What Is Scene Flow?**
- **Definition**: Dense 3D motion field estimated from stereo, RGB-D, or multi-view temporal input.
- **Difference from Optical Flow**: Optical flow gives 2D image displacement only.
- **Required Signals**: Depth and camera geometry are needed to recover true 3D movement.
- **Output Usage**: Dynamic scene understanding for robotics and autonomous driving.
**Why Scene Flow Matters**
- **Physical Motion Awareness**: Captures forward and backward depth movement, not just lateral pixel shift.
- **Planning Support**: Better inputs for collision prediction and trajectory planning.
- **3D Tracking**: Improves object motion estimation in world coordinates.
- **Sensor Fusion Value**: Bridges camera and depth modalities in one representation.
- **High-Stakes Utility**: Critical for safety-sensitive perception stacks.
**Scene Flow Pipeline**
**Geometry Estimation**:
- Recover depth or disparity from stereo or depth sensor.
- Convert pixels to 3D point representations.
**Temporal Correspondence**:
- Match points across time with learned or geometric correspondence methods.
- Estimate 3D displacement vectors per point.
**Consistency Regularization**:
- Enforce geometric and temporal consistency constraints.
- Reduce noise and occlusion-induced errors.
**How It Works**
**Step 1**:
- Compute frame-wise geometry and extract point or voxel features.
**Step 2**:
- Predict point correspondences and 3D displacement, then refine with consistency losses.
Scene flow estimation is **the 3D motion representation that turns image dynamics into physically meaningful movement understanding** - it is essential when systems must reason in real-world coordinates, not only image space.