Graph Convolution Gnns Message Passing
# Graph Convolution: GNNs & Message Passing
## Introduction & Motivation
Graph neural networks: learn on graph-structured data. Message passing: propagate information through edges. Graph convolution: convolve on graphs. Applications: social networks, molecular graphs, knowledge graphs, recommendation systems.
Motivation: Non-euclidean data; graphs ubiquitous. Specialized layers for graph structure.
Applications: Graphs, molecular, social networks.
---
## Core Concepts & Theory
### Message Passing
Nodes aggregate neighbor information.
### Graph Convolution
Spectral or spatial convolution.
### Node Classification
Predict node labels.
---
## Mathematical Formulation
Message passing (spatial):
$$x_i^{(k+1)} = \gamma^{(k)}(x_i^{(k)}, \square_{j \in N(i)} \phi^{(k)}(x_i^{(k)}, x_j^{(k)}, e_{ij}))$$
Graph convolution (spectral):
$$H^{(l+1)} = \sigma( ilde{D}^{-1/2} ilde{A} ilde{D}^{-1/2} H^{(l)} W^{(l)})$$
where à = A + I, D = degree matrix.
---
## Advanced Theory & Extensions
### Attention Mechanisms
Graph attention networks; GAT.
### Pooling
Graph pooling; node clustering.
### Heterogeneous Graphs
Multiple node/edge types; RGCN.
---
## Computational Considerations
GCN: O(|E| · D²) sparse multiplication.
Message Passing: O(|E|) aggregation.
Attention: O(|V|²) full attention.
---
## Practical Implementation Strategies
### Sparse Representations
Store only edges; memory efficient.
### Mini-Batch Training
Subgraph sampling; scalability.
### Normalization
Graph-level normalization; training stability.
---
## Benchmark Datasets & Evaluation
Cora: Citation network; node classification.
Citeseer: Publication network; classification.
OGB: Large-scale graphs; benchmarks.
---
## Key Challenges & Limitations
### Scalability
Large graphs challenging; sampling needed.
### Over-Smoothing
Deep GNNs lose node distinctiveness.
### Heterogeneity
Graphs have diverse node/edge types.
---
## Hyperparameter Tuning
Number of layers: 2-4 typical; avoid over-smoothing.
Hidden dimension: 64-512; model capacity.
Dropout: 0.1-0.5; regularization.
---
## Real-World Applications & Case Studies
Social Networks: Link prediction, community detection.
Molecular: Property prediction; drug discovery.
Knowledge Graphs: Link prediction; reasoning.
---
## Integration with Other Methods
GNN + Attention → attention-based aggregation.
GNN + Clustering → hierarchical graphs.
---
## Summary & Key Takeaways
Graph neural networks via message passing and graph convolution enable learning on graph-structured data through node aggregation and spectral operations.
Principles:
1. Message passing: neighbor aggregation.
2. Graph convolution: spectral filtering.
3. Node classification: per-node prediction.
4. Over-smoothing: depth limitation.
5. Scalability: sampling strategies.
---
---
## Appendix: Practical Labs
### Lab 1: Message Passing
import numpy as np
def message_passing_step(node_features, adjacency_matrix):
"""Single message passing step"""
# Aggregate messages from neighbors
aggregated = np.dot(adjacency_matrix, node_features)
return aggregated
# Test
np.random.seed(42)
num_nodes = 5
node_features = np.random.randn(num_nodes, 8)
adjacency = np.array([
[0, 1, 1, 0, 0],
[1, 0, 1, 1, 0],
[1, 1, 0, 1, 1],
[0, 1, 1, 0, 1],
[0, 0, 1, 1, 0]
], dtype=float)
aggregated = message_passing_step(node_features, adjacency)
assert aggregated.shape == node_features.shape, "Shape preserved"
print("✓ Message passing working")
if __name__ == "__main__":
print("Lab 1: MessagePassing - PASSED")### Lab 2: Graph Convolution Layer
import numpy as np
def graph_convolution(node_features, adjacency, weight):
"""Graph convolution layer"""
# Add self-loops
A_hat = adjacency + np.eye(len(adjacency))
# Compute degree matrix
degrees = A_hat.sum(axis=1)
D_inv_sqrt = np.diag(1 / np.sqrt(degrees + 1e-8))
# Normalized adjacency
A_norm = D_inv_sqrt @ A_hat @ D_inv_sqrt
# Graph convolution
output = A_norm @ node_features @ weight
return output
# Test
np.random.seed(42)
node_features = np.random.randn(5, 8)
adjacency = np.array([
[0, 1, 1, 0, 0],
[1, 0, 1, 1, 0],
[1, 1, 0, 1, 1],
[0, 1, 1, 0, 1],
[0, 0, 1, 1, 0]
], dtype=float)
weight = np.random.randn(8, 16)
output = graph_convolution(node_features, adjacency, weight)
assert output.shape == (5, 16), "Output shape"
print("✓ Graph convolution working")
if __name__ == "__main__":
print("Lab 2: GraphConv - PASSED")### Lab 3: Node Classification
import numpy as np
def node_classification_evaluate(predictions, true_labels, mask):
"""Evaluate node classification on masked nodes"""
masked_pred = predictions[mask]
masked_true = true_labels[mask]
accuracy = (masked_pred.argmax(axis=1) == masked_true).mean()
return accuracy
# Test
np.random.seed(42)
predictions = np.random.rand(10, 3) # 10 nodes, 3 classes
true_labels = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2, 0])
mask = np.array([False, True, True, False, True, True, False, True, True, False])
accuracy = node_classification_evaluate(predictions, true_labels, mask)
assert 0 <= accuracy <= 1, "Accuracy in [0,1]"
print("✓ Node classification evaluation working")
if __name__ == "__main__":
print("Lab 3: NodeClassification - PASSED")### Lab 4: Graph Metrics
import numpy as np
def compute_graph_statistics(adjacency_matrix):
"""Compute graph statistics"""
num_nodes = len(adjacency_matrix)
num_edges = adjacency_matrix.sum() / 2 # Undirected
degrees = adjacency_matrix.sum(axis=1)
avg_degree = degrees.mean()
# Density
max_edges = num_nodes * (num_nodes - 1) / 2
density = num_edges / max_edges if max_edges > 0 else 0
return {
"num_nodes": num_nodes,
"num_edges": int(num_edges),
"avg_degree": avg_degree,
"density": density
}
# Test
np.random.seed(42)
adjacency = np.random.rand(10, 10) > 0.7
adjacency = (adjacency + adjacency.T) / 2 # Symmetric
np.fill_diagonal(adjacency, 0)
stats = compute_graph_statistics(adjacency)
assert stats["num_nodes"] == 10, "Num nodes"
assert stats["num_edges"] >= 0, "Num edges"
print("✓ Graph metrics working")
if __name__ == "__main__":
print("Lab 4: GraphMetrics - PASSED")