Graph Neural Networks (GNNs) are the deep learning architectures that operate on graph-structured data (nodes connected by edges) — learning node, edge, and graph-level representations through iterative message passing where each node aggregates feature information from its neighbors, enabling tasks such as node classification, link prediction, and graph classification on social networks, molecular structures, knowledge graphs, and chip interconnect topologies that cannot be naturally represented as grids or sequences.

The Message Passing Framework

All GNNs follow a general message passing pattern: 1. Message: Each node computes a message to each neighbor based on its current features and the edge features: m_ij = MSG(h_i, h_j, e_ij). 2. Aggregation: Each node aggregates all incoming messages: a_i = AGG({m_ji : j ∈ N(i)}). AGG must be permutation-invariant (sum, mean, max). 3. Update: Node representation is updated: h_i' = UPDATE(h_i, a_i). 4. Repeat: Stack K message passing layers — each layer expands the receptive field by one hop. After K layers, each node's representation encodes information from its K-hop neighborhood.

Key GNN Architectures

• GCN (Graph Convolutional Network, Kipf & Welling): Symmetric normalized adjacation: h_i' = σ(Σ_j (1/√(d_i × d_j)) × W × h_j). Simple, effective, but uses fixed aggregation weights based on node degrees.
• GAT (Graph Attention Network): Attention coefficients α_ij = softmax(LeakyReLU(a^T [Wh_i || Wh_j])) determine how much node i attends to neighbor j. Adaptive aggregation — more informative neighbors get higher weight.
• GraphSAGE: Samples a fixed number of neighbors per node (avoids full neighborhood computation — enables training on large graphs). Aggregators: mean, LSTM, pooling.
• GIN (Graph Isomorphism Network): Maximally expressive message passing — provably as powerful as the Weisfeiler-Leman graph isomorphism test. Uses sum aggregation with MLP update: h_i' = MLP((1+ε) × h_i + Σ_j h_j).

Scalability Challenges

• Neighbor Explosion: A node with K-hop receptive field: if average degree is d, the K-hop neighborhood has d^K nodes. For K=3, d=50: 125,000 nodes per target node. Mini-batch training samples neighborhoods to bound computation.
• Full-Graph Methods: For the entire graph in GPU memory: GCN forward pass for N nodes, E edges, F features: O(E×F) per layer. Billion-edge graphs require distributed training or mini-batch sampling.

Applications in Hardware/EDA

• EDA Timing Prediction: Graph of circuit elements (gates, nets) — GNN predicts path delays, congestion, and power without running full static timing analysis. 100-1000× faster than traditional STA for initial exploration.
• Placement Optimization: Circuit netlist as a graph — GNN learns placement quality metrics. Google's chip design GNN generates floor plans for TPU blocks.
• Molecular Property Prediction: Atoms as nodes, bonds as edges — GNN predicts molecular properties (toxicity, solubility, binding affinity) for drug discovery.

Graph Neural Networks are the deep learning paradigm that extends neural networks beyond grids and sequences to arbitrary relational structures — enabling machine learning on the graph data that naturally represents most real-world systems from molecules to social networks to electronic circuits.