self-supervised depth and ego-motion
**Self-supervised depth and ego-motion learning** is the **joint training paradigm where depth and camera pose networks are optimized using view synthesis consistency instead of ground-truth labels** - it learns 3D scene structure and motion directly from raw video sequences.
**What Is Self-Supervised Depth and Ego-Motion?**
- **Definition**: Train depth predictor and pose estimator together by minimizing reprojection error across adjacent frames.
- **Supervision Source**: Geometry-induced photometric consistency rather than manual depth or pose annotation.
- **Typical Setup**: Target frame plus source frames with differentiable warping.
- **Popular Examples**: Monodepth-style pipelines with pose networks.
**Why It Matters**
- **Label-Free Scaling**: Uses abundant unlabeled videos for 3D learning.
- **Cost Reduction**: Avoids expensive depth sensor or motion-capture ground truth.
- **General Utility**: Provides initialization for SLAM, navigation, and reconstruction tasks.
- **Domain Adaptation**: Easier transfer to new environments using in-domain video.
- **Research Impact**: Major driver of modern monocular depth progress.
**Training Components**
**Depth Network**:
- Predict per-pixel depth for target frame.
- Often multi-scale outputs for stable training.
**Pose Network**:
- Predict relative transforms between target and source frames.
- Supports differentiable view warping.
**Photometric Loss Stack**:
- Reprojection error, smoothness terms, and masking for dynamic objects.
- Stabilizes training in non-ideal scenes.
**How It Works**
**Step 1**:
- Predict depth and relative pose, warp source frame into target view using differentiable geometry.
**Step 2**:
- Minimize reconstruction error plus regularization losses and update both networks jointly.
Self-supervised depth and ego-motion learning is **a scalable 3D perception strategy that turns video consistency into geometry and motion supervision** - it is a powerful alternative when labeled depth and pose data are scarce.