actor-critic methods
# Batch Normalization & Layer Normalization
## Introduction & Motivation
Normalization: stabilize training. Batch norm, layer norm, instance norm. Applications: deeper networks, faster convergence, improved generalization.
Motivation: Reduce internal covariate shift.
Applications: Deep CNNs, RNNs, Transformers.
---
## Core Concepts & Theory
### Batch Normalization
Normalize across batch.
### Layer Normalization
Normalize across features.
### Instance Normalization
Per-sample normalization.
### Group Normalization
Group-wise normalization.
---
## Mathematical Formulation
Batch Norm: \hat{x}_i = \frac{x_i - E[x_B]}{\sqrt{ ext{Var}(x_B) + \epsilon}}, \quad y_i = \gamma \hat{x}_i + \beta
Layer Norm: \hat{x} = \frac{x - ext{mean}(x)}{\sqrt{ ext{var}(x) + \epsilon}}
Instance Norm: \hat{x}_{nci} = \frac{x_{nci} - \mu_{ni}}{\sqrt{\sigma_{ni}^2 + \epsilon}}
---
## Advanced Theory & Extensions
### Batch Norm Variants
EvoNorm, FilterResponseNorm.
### Adaptive Normalization
Context-dependent scaling.
### Synchronized Batch Norm
Cross-GPU synchronization.
---
## Computational Considerations
Batch norm: O(batch_size·HW).
Layer norm: O(features).
Normalization: O(norm_dim).
---
## Practical Implementation Strategies
### Momentum Tracking
Running statistics for inference.
### Epsilon Tuning
Numerical stability.
### Affine Parameters
Learnable scale and shift.
---
## Benchmark Datasets & Evaluation
ImageNet: Batch norm effectiveness.
CIFAR-10: Normalization comparison.
GLUE: NLP normalization.
---
## Key Challenges & Limitations
### Small Batch Size
Noisy statistics.
### Train/Test Discrepancy
Different statistics.
### Synchronization Overhead
Distributed training cost.
---
## Hyperparameter Tuning
Momentum: 0.1-0.99.
Epsilon: 1e-5 to 1e-3.
Affine: True/False.
---
## Real-World Applications & Case Studies
Deep ResNets: Batch norm critical.
Transformer Training: Layer norm stability.
Distributed Training: Synchronized batch norm.
---
## Integration with Other Methods
Normalization + activation functions for stable gradients; + weight initialization for convergence.
---
## Summary & Key Takeaways
Normalization stabilizes deep network training.
Principles:
1. Batch norm: Reduce covariate shift.
2. Layer norm: Feature normalization.
3. Instance norm: Sample-level norm.
4. Group norm: Small batch solution.
5. Synchronized norm: Distributed training.
---
## Appendix: Practical Labs
### Lab 1: Batch Normalization
import numpy as np
def batch_norm(x, gamma, beta, eps=1e-5):
batch_mean = np.mean(x, axis=0)
batch_var = np.var(x, axis=0)
x_norm = (x - batch_mean) / np.sqrt(batch_var + eps)
y = gamma * x_norm + beta
return y
np.random.seed(42)
x = np.random.randn(32, 128)
gamma = np.ones(128)
beta = np.zeros(128)
y = batch_norm(x, gamma, beta)
assert y.shape == x.shape, "Correct normalized shape"
print("✓ Batch normalization working")### Lab 2: Layer Normalization
import numpy as np
def layer_norm(x, gamma, beta, eps=1e-5):
mean = np.mean(x, axis=-1, keepdims=True)
var = np.var(x, axis=-1, keepdims=True)
x_norm = (x - mean) / np.sqrt(var + eps)
y = gamma * x_norm + beta
return y
np.random.seed(42)
x = np.random.randn(32, 512)
gamma = np.ones(512)
beta = np.zeros(512)
y = layer_norm(x, gamma, beta)
assert y.shape == x.shape, "Correct normalized shape"
print("✓ Layer normalization working")### Lab 3: Instance Normalization
import numpy as np
def instance_norm(x, gamma, beta, eps=1e-5):
mean = np.mean(x, axis=(2, 3), keepdims=True)
var = np.var(x, axis=(2, 3), keepdims=True)
x_norm = (x - mean) / np.sqrt(var + eps)
y = gamma * x_norm + beta
return y
np.random.seed(42)
x = np.random.randn(4, 64, 32, 32)
gamma = np.ones((1, 64, 1, 1))
beta = np.zeros((1, 64, 1, 1))
y = instance_norm(x, gamma, beta)
assert y.shape == x.shape, "Correct normalized shape"
print("✓ Instance normalization working")### Lab 4: Group Normalization
import numpy as np
def group_norm(x, num_groups, gamma, beta, eps=1e-5):
batch_size, channels = x.shape[:2]
group_channels = channels // num_groups
x_reshaped = x.reshape(batch_size, num_groups, group_channels, -1)
mean = np.mean(x_reshaped, axis=(2, 3), keepdims=True)
var = np.var(x_reshaped, axis=(2, 3), keepdims=True)
x_norm = (x_reshaped - mean) / np.sqrt(var + eps)
x_norm = x_norm.reshape(x.shape)
y = gamma * x_norm + beta
return y
np.random.seed(42)
x = np.random.randn(4, 64, 32, 32)
gamma = np.ones(64)
beta = np.zeros(64)
y = group_norm(x, num_groups=8, gamma=gamma, beta=beta)
assert y.shape == x.shape, "Correct normalized shape"
print("✓ Group normalization working")---