geometric deep learning
**Geometric Deep Learning** is the **theoretical framework and set of architectures that incorporate geometric symmetries (translation, rotation, permutation, scale) as inductive biases into neural networks** — ensuring that if the input is transformed by a symmetry operation (e.g., rotated), the output transforms predictably (equivariance) or stays the same (invariance), leading to dramatically more data-efficient learning and physically correct predictions for molecular, protein, point cloud, and graph-structured data.
**Why Symmetry Matters**
- Standard MLP: No built-in symmetries → must learn rotation invariance from data (expensive).
- CNN: Built-in translation equivariance (feature map shifts with input shift).
- Geometric DL: Generalize this principle to ANY symmetry group.
```
Invariance: f(T(x)) = f(x) (output unchanged)
Equivariance: f(T(x)) = T'(f(x)) (output transforms correspondingly)
Example: Rotating a molecule → predicted energy stays the same (invariant)
Rotating a molecule → predicted forces rotate accordingly (equivariant)
```
**Symmetry Groups in Deep Learning**
| Group | Symmetry | Architecture | Application |
|-------|---------|-------------|-------------|
| Translation | Shift | CNN | Images |
| Permutation (Sₙ) | Reorder nodes | GNN | Graphs, sets |
| Rotation (SO(3)) | 3D rotation | SE(3)-equivariant nets | Molecules, proteins |
| Euclidean (SE(3)) | Rotation + translation | EGNN, PaiNN | Physics simulation |
| Scale | Zoom | Scale-equivariant CNN | Multi-resolution |
| Gauge (fiber bundle) | Local transformations | Gauge CNN | Manifolds |
**SE(3)-Equivariant Networks (Molecular/Protein AI)**
```python
# Equivariant Graph Neural Network (EGNN)
# Input: atom positions r_i, features h_i
# Output: updated positions and features that respect rotations
for layer in egnn_layers:
# Message: function of relative positions and features
m_ij = phi_e(h_i, h_j, ||r_i - r_j||²) # Distance is rotation-invariant
# Update positions: displacement along relative direction
r_i_new = r_i + Σ_j (r_i - r_j) * phi_x(m_ij) # Equivariant!
# Update features: aggregate messages
h_i_new = phi_h(h_i, Σ_j m_ij) # Invariant features
```
**Key Architectures**
| Architecture | Equivariance | Primary Use |
|-------------|-------------|-------------|
| SchNet | Translation + rotation invariant | Molecular energy |
| DimeNet | SO(3) invariant (angles + distances) | Molecular properties |
| PaiNN | SE(3) equivariant (scalar + vector) | Forces, dynamics |
| MACE | SE(3) equivariant (higher-order) | Molecular dynamics |
| SE(3)-Transformer | SE(3) equivariant attention | Protein structure |
| Equiformer | E(3) equivariant transformer | Molecular property |
**Impact: AlphaFold and Protein AI**
- AlphaFold2: Uses SE(3)-equivariant structure module.
- Invariant Point Attention: Attention that respects 3D rotational symmetry.
- Result: Atomic-accuracy protein structure prediction → Nobel Prize 2024.
- Without equivariance: Would need vastly more data and compute.
**Benefits of Geometric Priors**
| Metric | Non-equivariant | Equivariant | Improvement |
|--------|----------------|-------------|------------|
| Training data needed | 100K samples | 10K samples | 10× less |
| Generalization | Fails on rotated inputs | Perfect on rotated inputs | Correct by construction |
| Physics compliance | May violate conservation laws | Respects symmetries | Physically valid |
Geometric deep learning is **the principled framework for building neural networks that respect the fundamental symmetries of the physical world** — by incorporating group equivariance as an architectural constraint rather than something learned from data, geometric deep learning achieves superior data efficiency and physical correctness for molecular simulation, protein design, robotics, and any domain where the underlying physics has known symmetries.