Faster R-CNN - Region-Based Detection
# Faster R-CNN - Region-Based Detection
## Introduction & Motivation
Faster R-CNN: integrate RPN with R-CNN. End-to-end learnable region proposals. Applications: accurate object detection, video analysis.
Motivation: Eliminate separate region proposal computation.
Applications: Accurate object detection, video understanding, scene analysis.
---
## Core Concepts & Theory
### Region Proposal Network
Learnable proposal generation.
### Anchor Boxes
Predefined proposal templates.
### RoI Pooling
Extract fixed-size features from proposals.
### Multi-Task Loss
Combined proposal and detection.
---
## Mathematical Formulation
RPN Loss:
$$\mathcal{L}_{ ext{RPN}} = \mathcal{L}_{ ext{cls}}(p_i, p_i^*) + \lambda \mathcal{L}_{ ext{reg}}(t_i, t_i^*)$$
Detection Loss:
$$\mathcal{L}_{ ext{det}} = \mathcal{L}_{ ext{cls}} + \mathcal{L}_{ ext{box}}$$
RoI Pooling:
$$ ext{pool}(i,j) = \max_{(x,y) \in ext{bin}} f(x,y)$$
---
## Advanced Theory & Extensions
### Feature Pyramid Networks
Multi-scale detection.
### Cascade R-CNN
Multi-stage refinement.
### FPN-based Detection
Hierarchical region detection.
---
## Computational Considerations
RPN: O(H·W·A).
RoI pooling: O(N·h·w·D).
Detection head: O(N·C).
---
## Practical Implementation Strategies
### Anchor Matching
Assign anchors to GT boxes.
### Hard Negative Mining
Focus on difficult proposals.
### Batch Balancing
Balance positive/negative samples.
---
## Benchmark Datasets & Evaluation
COCO: Object detection benchmark.
Pascal VOC: Classic detection benchmark.
ImageNet: Large-scale detection.
---
## Key Challenges & Limitations
### Foreground-Background Imbalance
More negative anchors than positive.
### Small Object Detection
Limited feature resolution.
### Region Quality
Proposal quality affects detection.
---
## Hyperparameter Tuning
Anchor scales: 32, 64, 128, 256, 512.
RPN threshold: 0.7 (positive), 0.3 (negative).
Detection threshold: 0.05.
---
## Real-World Applications & Case Studies
Autonomous Driving: Vehicle and pedestrian detection.
Video Surveillance: Object tracking.
Medical Imaging: Lesion detection.
---
## Integration with Other Methods
Faster R-CNN + FPN for multi-scale; + cascade refinement.
---
## Summary & Key Takeaways
Faster R-CNN enables end-to-end region-based object detection.
Principles:
1. RPN: Learnable proposals.
2. Anchors: Predefined templates.
3. RoI pooling: Extract region features.
4. Multi-task: RPN + detection loss.
5. End-to-end: Joint optimization.
---
## Appendix: Practical Labs
### Lab 1: Region Proposal Network
import numpy as np
def rpn_forward(feature_map, anchors, num_proposals=2000):
"""Generate region proposals from RPN"""
h, w = feature_map.shape[:2]
# Dummy predictions
objectness = np.random.rand(h, w, len(anchors))
box_deltas = np.random.randn(h, w, len(anchors), 4) * 0.1
# Generate proposals
proposals = []
for i in range(h):
for j in range(w):
for a_idx in range(len(anchors)):
score = objectness[i, j, a_idx]
delta = box_deltas[i, j, a_idx]
proposal = anchors[a_idx] + delta
proposals.append((score, proposal))
# Sort by score and select top K
proposals = sorted(proposals, key=lambda x: x[0], reverse=True)[:num_proposals]
return proposals
np.random.seed(42)
feat = np.random.randn(32, 32, 256)
anchors = np.random.randn(9, 4)
proposals = rpn_forward(feat, anchors)
assert len(proposals) <= 2000
print(f"✓ RPN generated {len(proposals)} proposals")### Lab 2: Anchor-to-GT Assignment
import numpy as np
def assign_anchors_to_gt(anchors, gt_boxes, iou_threshold_pos=0.7, iou_threshold_neg=0.3):
"""Assign anchors to ground truth boxes"""
labels = np.zeros(len(anchors))
for i, anchor in enumerate(anchors):
# Compute IoU with all GT boxes
ious = []
for gt_box in gt_boxes:
iou = compute_iou(anchor, gt_box)
ious.append(iou)
max_iou = max(ious)
if max_iou >= iou_threshold_pos:
labels[i] = 1 # Positive
elif max_iou < iou_threshold_neg:
labels[i] = 0 # Negative
else:
labels[i] = -1 # Ignore
return labels
def compute_iou(box1, box2):
intersection = max(0, min(box1[2], box2[2]) - max(box1[0], box2[0])) * \
max(0, min(box1[3], box2[3]) - max(box1[1], box2[1]))
union = (box1[2]-box1[0])*(box1[3]-box1[1]) + (box1[2]-box1[0])*(box1[3]-box1[1]) - intersection
return intersection / union if union > 0 else 0
anchors = np.array([[10, 10, 50, 50], [100, 100, 150, 150], [200, 200, 250, 250]])
gt_boxes = np.array([[15, 15, 45, 45], [105, 105, 145, 145]])
labels = assign_anchors_to_gt(anchors, gt_boxes)
assert len(labels) == len(anchors)
print("✓ Anchor assignment working")### Lab 3: RoI Pooling
import numpy as np
def roi_pool(feature_map, rois, pool_size=(7, 7)):
"""Pool features for each region of interest"""
pooled = []
for roi in rois:
x1, y1, x2, y2 = roi.astype(int)
roi_feat = feature_map[y1:y2, x1:x2]
# Resize to pool_size (simplified)
h_stride = roi_feat.shape[0] / pool_size[0]
w_stride = roi_feat.shape[1] / pool_size[1]
pooled_roi = np.zeros((pool_size[0], pool_size[1], feature_map.shape[2]))
for i in range(pool_size[0]):
for j in range(pool_size[1]):
hi = int(i * h_stride)
hj = int(j * w_stride)
if hi < roi_feat.shape[0] and hj < roi_feat.shape[1]:
pooled_roi[i, j] = roi_feat[hi, hj]
pooled.append(pooled_roi)
return np.array(pooled)
np.random.seed(42)
feat = np.random.randn(224, 224, 256)
rois = np.array([[50, 50, 150, 150], [100, 100, 200, 200]])
pooled = roi_pool(feat, rois)
assert pooled.shape[1:] == (7, 7, 256)
print("✓ RoI pooling working")### Lab 4: RPN Loss
import numpy as np
def rpn_loss(objectness_pred, objectness_target, box_deltas, box_targets):
"""Compute RPN classification and regression loss"""
# Classification loss
cls_loss = -np.mean(objectness_target * np.log(objectness_pred + 1e-8) +
(1 - objectness_target) * np.log(1 - objectness_pred + 1e-8))
# Regression loss (Smooth L1)
diff = box_deltas - box_targets
reg_loss = np.mean(np.where(np.abs(diff) < 1, 0.5 * diff**2, np.abs(diff) - 0.5))
total = cls_loss + reg_loss
return total
np.random.seed(42)
obj_pred = np.random.rand(100)
obj_target = np.random.rand(100)
delta_pred = np.random.randn(100, 4)
delta_target = np.random.randn(100, 4)
loss = rpn_loss(obj_pred, obj_target, delta_pred, delta_target)
assert loss > 0
print(f"✓ RPN loss: {loss:.3f}")---