3d gaussian splatting
**3D Gaussian Splatting and AI-Driven 3D Generation** are the **techniques for creating and representing 3D scenes using collections of 3D Gaussian primitives (for reconstruction) and generative AI models (for text-to-3D creation)** — where Gaussian splatting achieves real-time novel view synthesis at 100+ FPS (100× faster than NeRF) by representing scenes as millions of colored 3D Gaussians that can be efficiently rasterized, and text-to-3D extends this by generating 3D assets from text descriptions using score distillation.
**3D Gaussian Splatting (3DGS)**
- Representation: Scene = collection of 3D Gaussians, each with:
- Position (μ): 3D center coordinates.
- Covariance (Σ): 3×3 matrix defining shape/orientation.
- Opacity (α): Transparency.
- Color (SH coefficients): View-dependent appearance using spherical harmonics.
- Rendering: Project 3D Gaussians onto 2D screen → alpha-blend in depth order → image.
- Key insight: Differentiable rasterization of Gaussians is 100-1000× faster than NeRF's ray marching.
**3DGS Pipeline**
```
Input: Multi-view photos of a scene (50-200 images)
↓
SfM (COLMAP): Estimate camera poses + sparse point cloud
↓
Initialize: One Gaussian per sparse point
↓
Optimize (gradient descent):
- Render from training camera poses
- Compare rendered image with ground truth (L1 + SSIM loss)
- Update Gaussian parameters (position, color, opacity, covariance)
- Adaptive density control: Split/clone/prune Gaussians
↓
Result: Scene with 500K-5M Gaussians, real-time rendering
```
**Performance Comparison**
| Method | Training Time | Rendering Speed | Quality (PSNR) |
|--------|-------------|----------------|----------------|
| NeRF (original) | 12-24 hours | 0.05 FPS | 31.0 dB |
| Instant-NGP | 5-10 minutes | 10-30 FPS | 33.2 dB |
| 3D Gaussian Splatting | 10-30 minutes | 100-300 FPS | 33.5 dB |
| Mip-Splatting | 15-40 minutes | 80-200 FPS | 33.8 dB |
**Text-to-3D Generation**
| Method | Approach | Speed | Quality |
|--------|---------|-------|--------|
| DreamFusion (Google) | SDS + NeRF | 1-2 hours | Good |
| Magic3D (NVIDIA) | Coarse-to-fine SDS | 40 min | High |
| GaussianDreamer | SDS + 3DGS | 15-25 min | Good |
| LGM | Feed-forward (no optimization) | 5 sec | Moderate |
| InstantMesh | Multi-view images → mesh | 10 sec | Good |
| Trellis | Latent 3D generation | 2-8 sec | High |
**Score Distillation Sampling (SDS)**
```
Optimize 3D representation θ so that:
Rendered images from any viewpoint "look good" to a pretrained 2D diffusion model
Gradient: ∇θ L_SDS ≈ E[w(t)(ε_φ(z_t; y, t) - ε) ∂x/∂θ]
- ε_φ: Pretrained diffusion model's noise prediction
- No need to backprop through diffusion model
- Works with any 3D representation (NeRF, mesh, Gaussians)
```
**Applications**
| Application | How 3DGS/Text-to-3D Is Used |
|------------|----------------------------|
| Gaming | Generate 3D assets from descriptions |
| Film VFX | Reconstruct real sets as digital twins |
| AR/VR | Photorealistic scene streaming |
| E-commerce | 3D product visualization from photos |
| Robotics | Build 3D maps for navigation |
| Architecture | Reconstruct buildings from drone footage |
3D Gaussian Splatting and text-to-3D generation are **revolutionizing how 3D content is created and rendered** — by replacing NeRF's slow ray marching with fast Gaussian rasterization and enabling 3D creation from text descriptions in seconds, these techniques are making high-quality 3D content creation accessible to anyone, fundamentally changing the economics of 3D asset production for games, film, VR, and digital commerce.