point cloud deep learning
**Point Cloud Deep Learning** encompasses **neural network architectures and techniques for processing 3D point cloud data — unordered sets of 3D coordinates (x,y,z) with optional attributes (color, normal, intensity)** — enabling applications in autonomous driving (LiDAR perception), robotics, 3D mapping, and industrial inspection where raw 3D data cannot be easily converted to regular grids or images.
**The Point Cloud Challenge**
```
Point cloud: {(x_i, y_i, z_i, features_i) | i = 1..N}
Key properties:
- Unordered: No canonical ordering (permutation invariant)
- Irregular: Non-uniform density, varying N
- Sparse: 3D space is mostly empty
- Large: LiDAR scans contain 100K-1M+ points
Cannot directly apply:
- CNNs (require regular grid)
- RNNs (require ordered sequence)
Need: architectures that handle unordered, variable-size 3D point sets
```
**PointNet (Qi et al., 2017): The Foundation**
```
Input: N×3 points (or N×D with features)
↓
Per-point MLP: shared weights, applied independently to each point
N×3 → N×64 → N×128 → N×1024
↓
Symmetric aggregation: MaxPool across all N points → 1×1024
(max pooling is permutation invariant!)
↓
Classification head: MLP → class probabilities
Segmentation head: concat global + per-point features → per-point labels
```
Key insight: **max pooling** is a symmetric function — invariant to point ordering. Per-point MLPs + global aggregation = universal set function approximator.
**PointNet++: Hierarchical Learning**
PointNet lacks local structure awareness. PointNet++ adds hierarchy:
```
Set Abstraction layers (like pooling in CNNs):
1. Farthest Point Sampling: select M << N center points
2. Ball Query: group neighbors within radius r for each center
3. Local PointNet: apply PointNet to each local group
→ M points with richer features
Repeat: hierarchical abstraction from N→M₁→M₂→... points
```
**Point Cloud Transformers**
| Model | Key Idea |
|-------|----------|
| PCT | Self-attention on point features, permutation invariant naturally |
| Point Transformer | Vector attention with subtraction (relative position) |
| Point Transformer V2 | Grouped vector attention, more efficient |
| Stratified Transformer | Stratified sampling for long-range + local |
Attention on points: Q_i = f(x_i), K_j = g(x_j), V_j = h(x_j) with positional encodings from 3D coordinates. Self-attention is naturally permutation-equivariant.
**Voxel and Hybrid Methods**
For large-scale outdoor scenes (autonomous driving):
- **VoxelNet**: Voxelize point cloud → 3D sparse convolution → dense BEV features
- **SECOND**: 3D sparse convolution (only compute at occupied voxels)
- **PV-RCNN**: Point-Voxel fusion — voxel features for proposals, point features for refinement
- **CenterPoint**: Detect 3D objects as center points in BEV
**Applications**
| Application | Task | Typical Architecture |
|------------|------|---------------------|
| Autonomous driving | 3D object detection | VoxelNet, CenterPoint |
| Robotics | Grasp detection, pose estimation | PointNet++, 6D pose |
| Indoor mapping | Semantic segmentation | Point Transformer |
| CAD/manufacturing | Shape classification, defect detection | DGCNN |
| Forestry/agriculture | Tree segmentation, terrain | RandLA-Net |
**Point cloud deep learning has matured from academic novelty to deployed industrial technology** — with architectures like PointNet establishing theoretical foundations and modern point transformers achieving state-of-the-art accuracy, 3D perception networks now power safety-critical autonomous systems processing millions of 3D points in real time.