neural scene flow

**Neural scene flow** is the **continuous 3D motion field learned by neural networks to map each scene point to its displacement over time** - it generalizes optical flow into metric 3D space and supports dynamic reconstruction, tracking, and motion reasoning. **What Is Neural Scene Flow?** - **Definition**: Implicit function that predicts 3D displacement vector for points given space and time coordinates. - **Input Form**: Coordinates, timestamp, and often latent scene features. - **Output Form**: Delta x, delta y, delta z motion vectors. - **Learning Signal**: Multi-view photometric consistency, geometric constraints, and temporal smoothness. **Why Neural Scene Flow Matters** - **Continuous Motion Model**: Avoids discrete correspondence limitations in sparse point matching. - **3D Dynamics**: Captures physically meaningful movement in world coordinates. - **Reconstruction Support**: Improves dynamic NeRF and 4D representation quality. - **Planning Utility**: Useful for robotics and autonomous perception of moving agents. - **Generalization**: Can represent complex non-rigid motion fields. **Modeling Patterns** **Implicit MLP Fields**: - Learn smooth motion function across space-time. - Flexible but may require strong regularization. **Feature-Conditioned Flow**: - Condition on latent geometry features for local detail. - Improves high-frequency motion fidelity. **Physics-Inspired Constraints**: - Add cycle consistency and smoothness terms. - Reduce implausible motion artifacts. **How It Works** **Step 1**: - Encode scene geometry and estimate initial correspondences across frames. **Step 2**: - Train neural flow field to minimize reprojection and temporal consistency errors. Neural scene flow is **the continuous motion representation that upgrades dynamic perception from 2D displacement to true 3D temporal geometry** - it is a key ingredient in modern 4D vision pipelines.