Graph Embeddings Node2vec Deepwalk Graph Representation

# Graph Embeddings: Node2Vec, DeepWalk & Graph Representation

## Introduction & Motivation

Graph Embeddings: learn node representations. Random walk-based methods. Applications: social networks, knowledge graphs, molecular graphs.

Motivation: Capture graph structure; downstream tasks.

Applications: Link prediction, node classification.

---

## Core Concepts & Theory

### Random Walks

Simulate graph traversals.

### Skip-gram

Word embedding applied to nodes.

### Node Proximity

First-order, second-order.

---

## Mathematical Formulation

Skip-gram objective:
$$L = -\sum_{u,v} \log P(v | u; heta)$$

Softmax:
$$P(v | u) = \frac{\exp(z_u^T z_v)}{\sum_w \exp(z_u^T z_w)}$$

---

## Advanced Theory & Extensions

### Node2Vec

Biased random walks; structural roles.

### DeepWalk

Pure random walk embedding.

### GraphSAGE

Inductive graph learning.

---

## Computational Considerations

Random walks: O(T·walk_len).

Skip-gram: O(context_size).

Training: O(walks·embedding_dim).

---

## Practical Implementation Strategies

### Walk Parameters

Walk length, return probability.

### Negative Sampling

Approximate softmax.

### Batch Training

Accelerated learning.

---

## Benchmark Datasets & Evaluation

Karate Club: Small social network.

Cora: Citation network.

BlogCatalog: Social network.

---

## Key Challenges & Limitations

### Scalability

Large graphs challenging.

### Inductive Setting

New nodes at test time.

### Dynamic Graphs

Changing structure over time.

---

## Summary & Key Takeaways

Graph Embeddings via random walks and skip-gram enable node representation learning from graph structure.

Principles:
1. Random walks: graph sampling.
2. Skip-gram: embedding learning.
3. Node2Vec: tunable proximity.
4. DeepWalk: scalable learning.
5. Applications: downstream tasks.

---

---

## Appendix: Practical Labs

### Lab 1: Random Walk

import numpy as np

def random_walk(graph, start_node, walk_length=10):
 """Generate random walk from node"""
 walk = [start_node]
 
 for _ in range(walk_length - 1):
 neighbors = graph[walk[-1]]
 if len(neighbors) == 0:
 break
 next_node = np.random.choice(neighbors)
 walk.append(next_node)
 
 return walk

# Test
np.random.seed(42)
graph = {0: [1, 2], 1: [0, 2, 3], 2: [0, 1], 3: [1]}

walk = random_walk(graph, 0, walk_length=5)

assert len(walk) <= 5, "Walk length"
print("✓ Random walk working")

if __name__ == "__main__":
 print("Lab 1: RandomWalk - PASSED")

### Lab 2: Skip-gram Loss

import numpy as np

def skip_gram_loss(embeddings, center_idx, context_idx):
 """Skip-gram loss for node embeddings"""
 z_c = embeddings[center_idx]
 z_t = embeddings[context_idx]
 
 # Cosine similarity
 sim = np.dot(z_c, z_t) / (np.linalg.norm(z_c) * np.linalg.norm(z_t) + 1e-8)
 
 # Loss (simplified)
 loss = -np.log(1 / (1 + np.exp(-sim)))
 
 return loss

# Test
np.random.seed(42)
embeddings = np.random.randn(10, 64)

loss = skip_gram_loss(embeddings, 0, 1)

assert np.isfinite(loss), "Loss finite"
print("✓ Skip-gram loss working")

if __name__ == "__main__":
 print("Lab 2: SkipGramLoss - PASSED")

### Lab 3: Node Similarity

import numpy as np

def node_similarity(embeddings, node_a, node_b):
 """Compute similarity between node embeddings"""
 z_a = embeddings[node_a]
 z_b = embeddings[node_b]
 
 # Cosine similarity
 sim = np.dot(z_a, z_b) / (np.linalg.norm(z_a) * np.linalg.norm(z_b) + 1e-8)
 
 return sim

# Test
np.random.seed(42)
embeddings = np.random.randn(10, 64)

sim = node_similarity(embeddings, 0, 1)

assert -1 <= sim <= 1, "Similarity in [-1,1]"
print("✓ Node similarity working")

if __name__ == "__main__":
 print("Lab 3: NodeSimilarity - PASSED")

### Lab 4: Link Prediction

import numpy as np

def predict_links(embeddings, similarity_threshold=0.5):
 """Predict missing links from embeddings"""
 n = len(embeddings)
 predicted_edges = []
 
 for i in range(n):
 for j in range(i + 1, n):
 # Cosine similarity
 z_i = embeddings[i]
 z_j = embeddings[j]
 sim = np.dot(z_i, z_j) / (np.linalg.norm(z_i) * np.linalg.norm(z_j) + 1e-8)
 
 if sim > similarity_threshold:
 predicted_edges.append((i, j))
 
 return predicted_edges

# Test
np.random.seed(42)
embeddings = np.random.randn(5, 64)

edges = predict_links(embeddings, 0.5)

assert isinstance(edges, list), "List of edges"
print("✓ Link prediction working")

if __name__ == "__main__":
 print("Lab 4: LinkPrediction - PASSED")