Advanced Neural 3D Representations encompasses the evolution beyond vanilla NeRF to faster, higher-quality neural 3D scene representations — including Instant-NGP's hash encoding for real-time training, 3D Gaussian Splatting's explicit point-based rendering, and hybrid approaches that have transformed neural 3D reconstruction from a research curiosity to a practical tool for content creation, mapping, and simulation.

NeRF Recap and Limitations

Original NeRF (2020) encodes a 3D scene as an MLP: f(x,y,z,θ,φ) → (color, density). Novel views are rendered by ray marching through the MLP. Limitations: hours to train, seconds to render a frame, struggles with large/dynamic scenes.

Instant-NGP (Multi-Resolution Hash Encoding)

NVIDIA's Instant-NGP (2022) achieved 1000× speedup over NeRF:

Input position (x,y,z)
    ↓
Multi-resolution hash grid: L levels, each with T hash entries
    Level 1: coarse grid → hash lookup → learnable feature vector
    Level 2: finer grid  → hash lookup → learnable feature vector
    ...
    Level L: finest grid → hash lookup → learnable feature vector
    ↓
Concatenate all level features → tiny MLP (2 layers) → color, density

Key innovations: (1) Hash table replaces dense grid — O(T) memory regardless of resolution; (2) Hash collisions are resolved by gradient-based learning; (3) Tiny MLP (65K parameters vs NeRF's 1.2M) — most representation power is in the hash table features; (4) Fully fused CUDA kernels. Result: 5-second training, real-time rendering.

3D Gaussian Splatting (3DGS)

3DGS (Kerbl et al., 2023) abandoned volumetric ray marching entirely for an explicit representation:

Scene = set of N 3D Gaussians, each with:
  - Position μ (3D center)
  - Covariance Σ (3D shape/orientation → 3×3 matrix, 6 params)
  - Color (spherical harmonics coefficients for view-dependent color)
  - Opacity α

Rendering: Project Gaussians to 2D → alpha-blend front-to-back
  (differentiable rasterization, NOT ray marching)

Why 3DGS is transformative:

• Explicit: No neural network evaluation per pixel — just project and splat
• Real-time: 100+ FPS at 1080p (vs. NeRF's seconds per frame)
• Editable: Move, delete, or modify individual Gaussians
• Fast training: 5-30 minutes (adaptive densification: clone/split/prune Gaussians during optimization)

Comparison

Active Research Frontiers

• Dynamic 3DGS: Deformable/temporal Gaussians for video (4D-GS, Dynamic3DGS)
• Compression: Reducing 3DGS storage from 100s of MB to <10 MB (compact-3DGS)
• Text-to-3D: DreamGaussian, LucidDreamer — generate 3D from text prompts using SDS
• Large-scale: City-scale reconstruction with hierarchical/tiled approaches
• SLAM: Gaussian splatting for real-time mapping and localization

Neural 3D representations have transitioned from research novelty to production-ready technology — with 3D Gaussian Splatting's real-time rendering and editability making neural 3D capture practical for applications ranging from VR content creation to autonomous driving simulation to digital twins.