Shape Completion is the computer vision task of predicting complete 3D geometry from partial observations — reconstructing missing surfaces, occluded regions, and unseen viewpoints from incomplete data such as single-view images, sparse depth maps, or partial point cloud scans — the enabling technology for robotics grasping of unseen object surfaces, autonomous driving scene understanding, and AR/VR environment reconstruction where sensors can never capture complete geometry in a single observation.

What Is Shape Completion?

• Definition: Given a partial 3D observation of an object or scene, predict the complete 3D shape including all surfaces not visible in the input — essentially hallucinating geometry that is geometrically plausible and semantically consistent with the observed portion.
• Input Modalities: Single-view RGB images, depth maps from RGB-D sensors, partial point clouds from LiDAR or structured light, incomplete mesh scans, or multi-view images with missing coverage.
• Output Representations: Completed voxel grids (occupancy), dense point clouds, signed distance fields (SDF), neural implicit functions (NeRF-style), or deformed mesh templates.
• Ambiguity Challenge: Shape completion is inherently ill-posed — multiple valid completions exist for any partial observation — requiring learned shape priors to resolve ambiguity.

Why Shape Completion Matters

• Robotic Grasping: Robots must plan grasps on surfaces they cannot see — shape completion predicts the back and underside of objects to enable stable grasp planning.
• Autonomous Driving: LiDAR captures only the facing surfaces of vehicles and pedestrians — completion enables full 3D bounding box estimation and occlusion reasoning.
• AR/VR Scene Reconstruction: Single scans of rooms have holes from occlusion and limited viewpoints — completion fills gaps to create watertight environments for immersive experiences.
• 3D Content Creation: Artists and designers can sketch partial shapes and let completion algorithms generate full 3D models — accelerating content pipelines.
• Medical Imaging: Partial organ scans from limited CT/MRI angles can be completed to full anatomical models for surgical planning.

Shape Completion Approaches

Voxel-Based Methods:

• Discretize 3D space into voxel grid; predict occupancy probability for each voxel.
• 3D convolutional encoder-decoder architectures (3D U-Net) process partial voxel input.
• Resolution limited by memory (128³ voxels requires ~2M parameters per layer).

Point Cloud Completion:

• Input: sparse or partial point cloud; output: dense, complete point cloud.
• Architectures: PointNet++ encoder with folding-based or coarse-to-fine decoder.
• PCN (Point Completion Network) and PoinTr (transformer-based) achieve state-of-the-art results.

Implicit Function Methods:

• Learn continuous occupancy or SDF functions: f(x,y,z) → occupancy/distance.
• Query at arbitrary resolution — not limited by voxel grid resolution.
• IF-Net, Occupancy Networks, and DeepSDF enable high-resolution completion.

Template Deformation:

• Start from category-specific mesh template; deform to match partial observation.
• Preserves mesh topology and enables texture transfer from template to completion.

Shape Completion Benchmarks

Shape Completion is the geometric imagination of computer vision — enabling machines to infer complete 3D structure from fragmentary observations, bridging the gap between what sensors can capture and what downstream tasks like manipulation, navigation, and reconstruction require to operate in the real world.