3D Gaussian Splatting and AI-Driven 3D Generation are the techniques for creating and representing 3D scenes using collections of 3D Gaussian primitives (for reconstruction) and generative AI models (for text-to-3D creation) — where Gaussian splatting achieves real-time novel view synthesis at 100+ FPS (100× faster than NeRF) by representing scenes as millions of colored 3D Gaussians that can be efficiently rasterized, and text-to-3D extends this by generating 3D assets from text descriptions using score distillation.

3D Gaussian Splatting (3DGS)

• Representation: Scene = collection of 3D Gaussians, each with:
• Position (μ): 3D center coordinates.
• Covariance (Σ): 3×3 matrix defining shape/orientation.
• Opacity (α): Transparency.
• Color (SH coefficients): View-dependent appearance using spherical harmonics.

• Rendering: Project 3D Gaussians onto 2D screen → alpha-blend in depth order → image.
• Key insight: Differentiable rasterization of Gaussians is 100-1000× faster than NeRF's ray marching.

3DGS Pipeline

Input: Multi-view photos of a scene (50-200 images)
         ↓
SfM (COLMAP): Estimate camera poses + sparse point cloud
         ↓
Initialize: One Gaussian per sparse point
         ↓
Optimize (gradient descent):
  - Render from training camera poses
  - Compare rendered image with ground truth (L1 + SSIM loss)
  - Update Gaussian parameters (position, color, opacity, covariance)
  - Adaptive density control: Split/clone/prune Gaussians
         ↓
Result: Scene with 500K-5M Gaussians, real-time rendering

Performance Comparison

Text-to-3D Generation

Score Distillation Sampling (SDS)

Optimize 3D representation θ so that:
  Rendered images from any viewpoint "look good" to a pretrained 2D diffusion model

Gradient: ∇θ L_SDS ≈ E[w(t)(ε_φ(z_t; y, t) - ε) ∂x/∂θ]
  - ε_φ: Pretrained diffusion model's noise prediction
  - No need to backprop through diffusion model
  - Works with any 3D representation (NeRF, mesh, Gaussians)

Applications

3D Gaussian Splatting and text-to-3D generation are revolutionizing how 3D content is created and rendered — by replacing NeRF's slow ray marching with fast Gaussian rasterization and enabling 3D creation from text descriptions in seconds, these techniques are making high-quality 3D content creation accessible to anyone, fundamentally changing the economics of 3D asset production for games, film, VR, and digital commerce.