depth estimation
**Monocular Depth Estimation** is the **computer vision task of predicting a dense depth map (distance from camera for every pixel) from a single RGB image** — a fundamentally ill-posed problem (infinite 3D scenes can produce the same 2D image) that deep learning has made practically solvable by learning depth cues from large-scale training data, enabling applications in autonomous driving, AR/VR, 3D photography, and robotics without requiring dedicated depth sensors.
**Types of Depth Estimation**
| Type | Input | Output | Hardware |
|------|-------|--------|----------|
| Stereo | Two cameras | Metric depth | Stereo camera pair |
| LiDAR | Laser scanner | Sparse metric depth | Expensive sensor |
| Structured Light | IR projector + camera | Dense depth | Depth sensor (RealSense) |
| Monocular | Single RGB image | Relative or metric depth | Any camera |
| Multi-View | Multiple images (same camera) | Dense depth | Single moving camera |
**Monocular Depth Approaches**
| Method | Training Data | Output Type |
|--------|-------------|------------|
| Supervised | RGB + ground-truth depth (LiDAR) | Metric depth |
| Self-supervised | Stereo image pairs or video | Relative depth |
| Zero-shot (foundation) | Large mixed datasets | Relative or metric depth |
**Key Models**
| Model | Year | Key Innovation |
|-------|------|---------------|
| Eigen et al. | 2014 | First deep monocular depth (multi-scale CNN) |
| Monodepth2 | 2019 | Self-supervised from monocular video |
| MiDaS | 2020 | Multi-dataset training → robust zero-shot |
| DPT | 2021 | Vision Transformer + dense prediction |
| Depth Anything (v1/v2) | 2024 | Foundation depth model, SOTA zero-shot |
| Metric3D v2 | 2024 | Metric depth from single image |
| UniDepth | 2024 | Camera-aware metric depth |
**Relative vs. Metric Depth**
- **Relative depth**: Correct ordering (A is closer than B) but unknown scale.
- Sufficient for: Image editing, relighting, bokeh effect.
- **Metric depth**: Actual distances in meters.
- Required for: Autonomous driving, robotics, AR placement.
- Challenge: A single image lacks absolute scale information.
- Solutions: Learn from metric datasets, use camera intrinsics as input.
**Depth Anything (Foundation Model)**
- Trained on 62M unlabeled images + 1.5M labeled images.
- Self-teaching: DINOv2 teacher provides pseudo-depth for unlabeled images.
- Robust zero-shot: Works on any domain (indoor, outdoor, medical, underwater).
- v2: Adds metric depth heads fine-tuned on specific domains.
**Applications**
| Application | How Depth Is Used |
|------------|-------------------|
| Portrait mode (phones) | Depth map → blur background (bokeh) |
| AR/VR occlusion | Virtual objects hidden behind real objects |
| Autonomous driving | Depth for obstacle detection without LiDAR |
| 3D photo/video | Convert 2D image to 3D for VR viewing |
| Robotics | Depth for grasping, navigation |
| Novel view synthesis | Depth-guided NeRF/3DGS initialization |
Monocular depth estimation is **one of the most practically impactful computer vision achievements** — by extracting 3D structure from ordinary 2D images, it enables depth-aware applications on every smartphone camera, making previously sensor-dependent capabilities universally accessible through software alone.