Mobilenet - Efficient Mobile Architecture

# MobileNet - Efficient Mobile Architecture

## Introduction & Motivation

MobileNet: lightweight CNNs for mobile/edge devices. Depthwise separable convolutions. Applications: mobile inference, embedded systems.

Motivation: Reduce model size for deployment.

Applications: Mobile app integration, edge computing.

---

## Core Concepts & Theory

### Depthwise Separable Convolution

Efficient spatial filtering.

### Width Multiplier

Channel reduction factor.

### Resolution Multiplier

Input resolution scaling.

### Computational Efficiency

Reduced FLOPs and parameters.

---

## Mathematical Formulation

Depthwise Convolution:
$$ ext{DW}(x)_{i,j,k} = \sum_m \sum_n w_{m,n,k} \cdot x_{i+m,j+n,k}$$

Pointwise Convolution:
$$ ext{PW}(x)_{i,j,c'} = \sum_c w_{c,c'} \cdot x_{i,j,c}$$

Computational Reduction:
$$\frac{ ext{DW+PW}}{ ext{Standard}} = \frac{1}{C_{ ext{out}}} + \frac{1}{K^2}$$

---

## Advanced Theory & Extensions

### MobileNetV2

Inverted bottleneck blocks.

### MobileNetV3

Hardware-aware optimization.

### Knowledge Distillation

Compress with teacher.

---

## Computational Considerations

Depthwise: O(H·W·C·K²).

Pointwise: O(H·W·C_in·C_out).

Total: ~1/8 to 1/9 of standard.

---

## Practical Implementation Strategies

### Width Scaling

Alpha parameter adjustment.

### Resolution Scaling

Input size reduction.

### Quantization

Integer inference.

---

## Benchmark Datasets & Evaluation

ImageNet: Classification benchmark.

Mobile Settings: Device latency.

FLOPS: Computational efficiency.

---

## Key Challenges & Limitations

### Accuracy Trade-off

Size vs. performance.

### Hardware Optimization

Device-specific tuning.

### Batch Norm Dependency

Training requirements.

---

## Hyperparameter Tuning

Width multiplier: 0.25-1.0.

Resolution multiplier: 0.5-1.0.

Learning rate: 1e-4 to 1e-3.

---

## Real-World Applications & Case Studies

Mobile Apps: On-device inference.

Edge Devices: IoT deployment.

Real-time Systems: Low-latency requirements.

---

## Integration with Other Methods

MobileNet + distillation; + quantization for further compression.

---

## Summary & Key Takeaways

MobileNet achieves efficient inference via depthwise separable convolutions.

Principles:
1. Depthwise separable: Reduce computation.
2. Width multiplier: Channel scaling.
3. Resolution multiplier: Input scaling.
4. Efficiency: Mobile optimization.
5. Quantization: Further compression.

---

## Appendix: Practical Labs

### Lab 1: Depthwise Convolution

import numpy as np

def depthwise_conv2d(x, w):
 """Depthwise convolution per-channel"""
 h, w_size = w.shape
 c = x.shape[2]
 out = np.zeros((x.shape[0] - h + 1, x.shape[1] - w_size + 1, c))
 
 for ch in range(c):
 for i in range(out.shape[0]):
 for j in range(out.shape[1]):
 patch = x[i:i+h, j:j+w_size, ch]
 out[i, j, ch] = np.sum(patch * w)
 
 return out

np.random.seed(42)
x = np.random.randn(28, 28, 3)
w = np.random.randn(3, 3)
out = depthwise_conv2d(x, w)
assert out.shape[2] == 3, "Correct channel count"
print("✓ Depthwise convolution working")

### Lab 2: Width Multiplier

import numpy as np

def apply_width_multiplier(num_channels, alpha):
 """Scale channels by width multiplier"""
 scaled = max(1, int(num_channels * alpha))
 return scaled

channels_v1 = 32
alpha = 0.5
channels_v2 = apply_width_multiplier(channels_v1, alpha)
assert channels_v2 == 16, "Correct scaling"
print(f"✓ Width multiplier: {channels_v1} → {channels_v2}")

### Lab 3: Computational Efficiency

import numpy as np

def compare_flops(h, w, c_in, c_out, k):
 """Compare standard vs. depthwise convolution"""
 standard = h * w * c_in * c_out * k * k
 
 depthwise = h * w * c_in * k * k
 pointwise = h * w * c_in * c_out
 separable = depthwise + pointwise
 
 ratio = separable / standard
 return ratio

h, w, c_in, c_out, k = 28, 28, 32, 64, 3
ratio = compare_flops(h, w, c_in, c_out, k)
assert ratio < 1, "Depthwise more efficient"
print(f"✓ Efficiency ratio: {ratio:.3f}")

### Lab 4: Resolution Scaling

import numpy as np

def apply_resolution_multiplier(input_size, rho):
 """Scale input resolution"""
 scaled_size = int(input_size * rho)
 return scaled_size

input_size = 224
rho = 0.75
scaled = apply_resolution_multiplier(input_size, rho)
assert scaled == 168, "Correct scaling"
print(f"✓ Resolution multiplier: {input_size} → {scaled}")

---