point cloud

**Point Cloud Deep Learning** is the **application of neural networks to 3D point cloud data** — unordered sets of (x,y,z) coordinates representing 3D scenes, enabling autonomous driving perception, robotic mapping, and 3D object recognition. **What Is a Point Cloud?** - Set of N points, each with coordinates $(x, y, z)$ and optional attributes (intensity, color, normal). - Generated by: LiDAR scanners, depth cameras (Intel RealSense), stereo vision, photogrammetry. - LiDAR: 16–128 beams, 100K–500K points per scan at 10Hz — primary sensor for autonomous driving. **Challenges vs. Images** - **Irregular structure**: Points are unordered — no fixed grid (unlike pixels). - **Sparsity**: Most 3D space is empty. - **Variable density**: Near objects: dense; far objects: sparse. - **No standard convolution**: Regular CNN needs grid — point clouds lack it. **PointNet (2017)** - First deep learning directly on point clouds. - Key insight: Symmetric function (max pooling) handles unordered sets. - Architecture: MLP on each point independently → Global max pool → classification head. - Transformation network (T-Net): Learn input/feature alignment. - Limitation: No local structure — every point treated globally. **PointNet++ (2017)** - Hierarchical grouping: Local neighborhoods → hierarchical features. - Sampling: Farthest point sampling (FPS) selects representative centroids. - Set Abstraction: MLP on neighborhood → local feature. - Captures both local and global structure. **Voxel-Based Methods** - VoxelNet: Quantize points to voxels → 3D CNN. - PointPillars: Pillar (vertical column) features → 2D pseudo-image → 2D CNN. - Real-time: 62 FPS, competitive accuracy — standard for production AV. **Transformer-Based** - Point Transformer: Self-attention with local neighborhoods. - PCT (Point Cloud Transformer): Global self-attention on point features. Point cloud deep learning is **the critical perception technology for autonomous systems** — enabling LiDAR-based obstacle detection, lane understanding, and 3D map building that complements camera-based vision for all-weather reliable autonomous navigation.