FlowNet is the pioneering end-to-end deep optical flow architecture that predicts dense motion directly from image pairs - it demonstrated that learned correspondence can replace many handcrafted motion-estimation stages.

What Is FlowNet?

• Definition: Convolutional encoder-decoder model for optical flow estimation from two frames.
• Original Variants: FlowNetS (stacked input) and FlowNetC (with correlation layer).
• Output: Dense per-pixel flow map at full or near-full resolution.
• Historical Impact: First major deep model family to make neural flow practical.

Why FlowNet Matters

• End-to-End Learning: Removed dependence on manual feature engineering in flow pipelines.
• Speed Gain: Enabled near real-time flow inference on GPUs at the time.
• Benchmark Shift: Sparked rapid progress in learned correspondence models.
• Architecture Foundation: Influenced PWC-Net, RAFT, and later refinement approaches.
• Practical Utility: Widely reused as initialization and baseline in motion tasks.

FlowNet Architecture Highlights

Encoder-Decoder Core:

• Downsample to capture large-context matching cues.
• Upsample with skip connections for fine motion recovery.

Correlation Module (FlowNetC):

• Explicitly compares feature patches between two frames.
• Improves matching quality for structured motion.

Multi-Scale Supervision:

• Predict flow at several scales and refine progressively.
• Stabilizes training and detail recovery.

How It Works

Step 1:

• Feed frame pair through convolutional encoder to build feature hierarchy and correspondence cues.

Step 2:

• Decode coarse-to-fine flow estimates with skip fusion to produce dense final output.

FlowNet is the foundational deep optical-flow milestone that proved neural networks can learn pixel correspondence directly from data - it remains an essential reference point in modern motion estimation history.