neural radiance field nerf

**Neural Radiance Fields (NeRF)** is **the 3D scene representation that encodes a continuous volumetric scene as a neural network mapping 3D coordinates and viewing direction to color and density — enabling photorealistic novel view synthesis from a sparse set of input photographs through differentiable volume rendering**. **NeRF Representation:** - **Implicit Function**: F(x,y,z,θ,φ) → (r,g,b,σ) maps spatial position (x,y,z) and viewing direction (θ,φ) to color (RGB) and volume density (σ); the neural network (typically 8-layer MLP with 256 hidden units) represents the entire scene as a continuous function - **View-Dependent Color**: color depends on viewing direction to model specular reflections and view-dependent appearance; density depends only on position (geometry is view-independent); this separation is architecturally enforced by feeding direction only to later MLP layers - **Positional Encoding**: raw coordinates are transformed via sinusoidal functions γ(x) = [sin(2⁰πx), cos(2⁰πx), ..., sin(2^(L-1)πx), cos(2^(L-1)πx)] with L=10 for position and L=4 for direction; without positional encoding, the MLP cannot learn high-frequency geometric and appearance details - **Scene Bounds**: NeRF assumes a bounded scene; ray sampling is distributed within the scene bounds; unbounded scenes require specialized parameterization (mip-NeRF 360) that contracts distant regions into a bounded volume **Volume Rendering:** - **Ray Marching**: for each pixel, cast a ray from the camera through the image plane; sample N points (64 coarse + 64 fine) along the ray within the scene bounds; evaluate the MLP at each sample point to obtain (color, density) - **Alpha Compositing**: pixel color C(r) = Σ_i T_i·α_i·c_i where α_i = 1-exp(-σ_i·δ_i), T_i = Π_{j100 fps at 1080p) through GPU-optimized splatting - **Adaptive Density**: Gaussians are cloned (split large) and pruned (remove transparent) during training to adaptively adjust point density where scene complexity demands it; starts from SfM point cloud and densifies to capture fine details - **Quality vs Speed**: matches or exceeds NeRF quality for novel view synthesis with 100-1000× faster rendering; enables VR/AR applications, game engine integration, and real-time scene exploration NeRF and 3D Gaussian Splatting represent **the revolution in neural 3D reconstruction — transforming sparse photographs into photorealistic, explorable 3D scenes, enabling applications from virtual reality to autonomous driving simulation to digital heritage preservation**.