Compound Scaling is the principled method of jointly scaling a neural network's depth, width, and resolution using a fixed ratio — introduced in EfficientNet, showing that balanced scaling outperforms scaling any single dimension.

How Does Compound Scaling Work?

• Three Dimensions: Depth ($d$), Width ($w$), Resolution ($r$).
• Constraint: $alpha cdot eta^2 cdot gamma^2 approx 2$ (doubles FLOPs per unit increase in $phi$).
• Grid Search: Find optimal $alpha, eta, gamma$ on a small model (B0). Then scale with $phi$.
• Result: $d = alpha^phi, w = eta^phi, r = gamma^phi$.

Why It Matters

• Balanced Growth: Networks that only grow deeper (ResNet-1000) or only wider (Wide-ResNet) hit diminishing returns. Compound scaling avoids this.
• Universal: The principle applies beyond EfficientNet — any architecture benefits from balanced scaling.
• Design Rule: Provides a concrete recipe for scaling any base architecture.

Compound Scaling is the growth formula for neural networks — a mathematical recipe ensuring balanced development across all dimensions.