video

**Video Understanding Temporal Models** is **neural architectures capturing temporal dynamics in video sequences, enabling action recognition, temporal localization, and event understanding from continuous visual information** — extends image understanding to sequences. Temporal modeling essential for video tasks. **3D Convolution** extends 2D convolution to temporal dimension. 3D filters convolve over (height, width, time). Captures spatiotemporal features—motion, transitions, actions. Computationally expensive (larger filters, more parameters) than 2D. **Two-Stream Architecture** two pathways: spatial stream processes individual frames (appearance), temporal stream processes optical flow (motion). Fusion combines streams. Separates appearance and motion learning. **Optical Flow** estimates pixel motion between frames. Used directly as input to temporal stream or computed features. Lucas-Kanade, FlowNet (CNN-based). **Recurrent Neural Networks for Video** LSTMs process frame sequences, capturing temporal dependencies through recurrence. Hidden state carries information across frames. Can process variable-length videos. **Temporal Segment Networks** divide video into segments, sample frames from each segment, classify each segment, aggregate predictions. Captures temporal structure. **Attention Mechanisms** temporal attention weights different frames when making decisions. Learns which frames are important for task. Spatial attention weights regions within frames. **Transformer Models** self-attention attends to all frames simultaneously. Positional encodings for temporal position. Computationally expensive for long videos. Can use sparse attention (restrict attention spatially/temporally). **Action Localization (Temporal)** identify start and end times of actions in untrimmed videos. Region proposal networks adapted for temporal dimension. Two-stage: generate candidates, classify candidates. **Slowfast Networks** dual-pathway architecture: slow pathway (low frame rate, low temporal resolution, high semantic information), fast pathway (high frame rate, detailed temporal information). Fused for action recognition. **Video Classification** classify entire video into action class. Aggregation: average pool, attention-weighted, recurrent. **Datasets and Benchmarks** Kinetics-400/700 (large-scale action recognition), Something-Something (temporal reasoning), UCF101, HMDB51 (smaller benchmarks). **Optical Flow Networks** FlowNet learns to estimate flow end-to-end. PWCNet, RAFT improve accuracy. Unsupervised learning from photometric loss. **RGB and Flow Fusion** combining appearance (RGB) and motion (flow) improves accuracy. Late fusion: separate classifiers fused post-hoc. Early fusion: combined features. **Temporal Reasoning** Some videos require causal reasoning. Temporal convolutions or transformers capture causes preceding effects. **Instance Segmentation in Video** temporally coherent segmentation masks. Tracking-by-detection or optical flow propagation. **Streaming Video Understanding** process video frame-by-frame as it arrives. Challenge: decisions based on incomplete information. Sliding window buffer. **Efficiency** video inherently redundant across frames. Frame subsampling without accuracy loss. Compressed representations (keyframes). **Applications** action recognition (sports analytics, surveillance), video recommendation, autonomous driving (activity detection in scenes), video retrieval. **Multimodal Video Understanding** combining audio and visual information improves understanding. Synchronization critical. **Domain Adaptation** models trained on one action dataset transfer poorly to others (domain gap). Unsupervised domain adaptation techniques. **Video understanding models enable automated analysis of video content** critical for surveillance, recommendation, embodied AI.