Neural Radiance Fields (NeRF) is the 3D scene representation that encodes a continuous volumetric scene as a neural network mapping 3D coordinates and viewing direction to color and density — enabling photorealistic novel view synthesis from a sparse set of input photographs through differentiable volume rendering.

NeRF Representation:

• Implicit Function: F(x,y,z,θ,φ) → (r,g,b,σ) maps spatial position (x,y,z) and viewing direction (θ,φ) to color (RGB) and volume density (σ); the neural network (typically 8-layer MLP with 256 hidden units) represents the entire scene as a continuous function
• View-Dependent Color: color depends on viewing direction to model specular reflections and view-dependent appearance; density depends only on position (geometry is view-independent); this separation is architecturally enforced by feeding direction only to later MLP layers
• Positional Encoding: raw coordinates are transformed via sinusoidal functions γ(x) = [sin(2⁰πx), cos(2⁰πx), ..., sin(2^(L-1)πx), cos(2^(L-1)πx)] with L=10 for position and L=4 for direction; without positional encoding, the MLP cannot learn high-frequency geometric and appearance details
• Scene Bounds: NeRF assumes a bounded scene; ray sampling is distributed within the scene bounds; unbounded scenes require specialized parameterization (mip-NeRF 360) that contracts distant regions into a bounded volume

Volume Rendering:

• Ray Marching: for each pixel, cast a ray from the camera through the image plane; sample N points (64 coarse + 64 fine) along the ray within the scene bounds; evaluate the MLP at each sample point to obtain (color, density)
• Alpha Compositing: pixel color C(r) = Σ_i T_i·α_i·c_i where α_i = 1-exp(-σ_i·δ_i), T_i = Π_{j
• Hierarchical Sampling: coarse network samples uniformly; fine network samples more densely around surfaces detected by the coarse network (high-density regions); improves detail without wasting samples in empty space
• Differentiable Rendering: the entire rendering pipeline from ray → neural network → pixel color is differentiable; reconstruction loss |C_rendered - C_target|² provides supervision from 2D photographs

Speed Improvements:

• Instant-NGP (Hash Encoding): replaces the slow MLP positional encoding with a multi-resolution hash grid; spatial positions are hashed into learned feature tables at multiple resolutions; training time reduced from hours to seconds; inference enables real-time rendering (~60 fps)
• TensoRF: decomposes the radiance field into low-rank tensor components (vector-matrix factorization); 100× faster than original NeRF with comparable quality; memory-efficient representation enabling larger scenes
• Plenoxels: replaces the MLP entirely with a sparse voxel grid storing spherical harmonic coefficients; training via direct optimization of voxel values; removes neural network overhead entirely but requires more memory
• Baked NeRF: pre-computes the neural field into a discrete representation (sparse voxel grid, textured mesh) for real-time rendering; trades storage for interactive rendering speed

3D Gaussian Splatting:

• Explicit 3D Representation: represents the scene as millions of 3D Gaussian primitives, each with position, covariance (shape/orientation), opacity, and spherical harmonic color coefficients; fundamentally different from NeRF's implicit MLP representation
• Rasterization Pipeline: projects 3D Gaussians to 2D screen space and alpha-blends in depth order; differentiable rasterization enables end-to-end training from photographs; achieves real-time rendering (>100 fps at 1080p) through GPU-optimized splatting
• Adaptive Density: Gaussians are cloned (split large) and pruned (remove transparent) during training to adaptively adjust point density where scene complexity demands it; starts from SfM point cloud and densifies to capture fine details
• Quality vs Speed: matches or exceeds NeRF quality for novel view synthesis with 100-1000× faster rendering; enables VR/AR applications, game engine integration, and real-time scene exploration

NeRF and 3D Gaussian Splatting represent the revolution in neural 3D reconstruction — transforming sparse photographs into photorealistic, explorable 3D scenes, enabling applications from virtual reality to autonomous driving simulation to digital heritage preservation.