Quantization Techniques

# Convolutional Neural Networks Architecture

## Introduction & Motivation

CNNs: process spatial data. Convolution, pooling, feature maps. Applications: image recognition, object detection.

Motivation: Exploit spatial structure in images.

Applications: Computer vision tasks.

---

## Core Concepts & Theory

### Convolution Operation

Local feature extraction.

### Pooling

Dimension reduction.

### Feature Maps

Learned representations.

### Receptive Fields

Context aggregation.

---

## Mathematical Formulation

Convolution: y[i,j] = \sum_m \sum_n w[m,n] \cdot x[i+m, j+n] + b

Pooling: y[i,j] = \max(x[i:i+k, j:j+k])

Receptive Field: RF_{\ell} = RF_{\ell-1} + (K_\ell - 1) \prod_i S_i

---

## Advanced Theory & Extensions

### Dilated Convolution

Increased receptive field.

### Grouped Convolution

Reduced computation.

### Depthwise Separable

Efficient convolution.

---

## Computational Considerations

Convolution: O(H·W·C_in·C_out·K²).

Pooling: O(H·W·K²).

Flattening: O(H·W·C).

---

## Practical Implementation Strategies

### Kernel Size Selection

Reception vs. computation.

### Stride Tuning

Downsampling control.

### Padding

Boundary handling.

---

## Benchmark Datasets & Evaluation

ImageNet: Standard CNN benchmark.

CIFAR-10: Small-scale comparison.

MNIST: Baseline validation.

---

## Key Challenges & Limitations

### Computational Cost

Heavy computation.

### Memory Usage

Feature map storage.

### Parameter Count

Millions of parameters.

---

## Hyperparameter Tuning

Kernel size: 3x3 to 7x7.

Stride: 1-2.

Padding: 'same' or 'valid'.

---

## Real-World Applications & Case Studies

Image Classification: CNN standard.

Object Detection: YOLO, R-CNN.

Semantic Segmentation: Fully convolutional networks.

---

## Integration with Other Methods

CNNs + attention for selective focus; + ResNets for deep architectures.

---

## Summary & Key Takeaways

CNNs leverage spatial structure for image understanding.

Principles:
1. Convolution: Local feature learning.
2. Pooling: Dimension reduction.
3. Stacking: Hierarchical features.
4. Receptive fields: Context aggregation.
5. Efficiency: Parameter sharing.

---

## Appendix: Practical Labs

### Lab 1: 2D Convolution

import numpy as np

def convolve2d(x, w, padding=0, stride=1):
 h, w_size = w.shape
 h_img, w_img = x.shape[:2]
 
 h_out = (h_img + 2*padding - h) // stride + 1
 w_out = (w_img + 2*padding - w_size) // stride + 1
 
 out = np.zeros((h_out, w_out))
 
 for i in range(h_out):
 for j in range(w_out):
 h_start = i * stride
 w_start = j * stride
 patch = x[h_start:h_start+h, w_start:w_start+w_size]
 out[i, j] = np.sum(patch * w)
 
 return out

np.random.seed(42)
x = np.random.randn(28, 28)
w = np.random.randn(3, 3)
out = convolve2d(x, w)
assert out.shape[0] <= x.shape[0], "Output height reduced"
print("✓ 2D convolution working")

### Lab 2: Max Pooling

import numpy as np

def max_pool2d(x, pool_size=2, stride=2):
 h, w = x.shape
 h_out = (h - pool_size) // stride + 1
 w_out = (w - pool_size) // stride + 1
 
 out = np.zeros((h_out, w_out))
 
 for i in range(h_out):
 for j in range(w_out):
 h_start = i * stride
 w_start = j * stride
 patch = x[h_start:h_start+pool_size, w_start:w_start+pool_size]
 out[i, j] = np.max(patch)
 
 return out

np.random.seed(42)
x = np.random.randn(28, 28)
out = max_pool2d(x, pool_size=2)
assert out.shape == (14, 14), "Correct pooling shape"
print("✓ Max pooling working")

### Lab 3: Receptive Field

import numpy as np

def calculate_receptive_field(layer_config):
 rf = 1
 for kernel_size, stride in layer_config:
 rf = rf + (kernel_size - 1) * stride
 return rf

layer_config = [(3, 1), (3, 1), (3, 1), (3, 2)]
rf = calculate_receptive_field(layer_config)
assert rf > 1, "Receptive field computed"
print(f"✓ Receptive field: {rf}")

### Lab 4: Feature Maps

import numpy as np

def visualize_feature_maps(feature_maps):
 num_filters = feature_maps.shape[2]
 stats = {}
 for i in range(num_filters):
 fm = feature_maps[:, :, i]
 stats[f'filter_{i}'] = {
 'mean': np.mean(fm),
 'std': np.std(fm),
 'max': np.max(fm)
 }
 return stats

np.random.seed(42)
feature_maps = np.random.randn(28, 28, 64)
stats = visualize_feature_maps(feature_maps)
assert len(stats) == 64, "Correct filter count"
print("✓ Feature map visualization working")

---