Optical Flow: Principles, Methods, and Applications in Computer Vision

Authors : Huang Zhibiao (Master's from University of Chinese Academy of Sciences, JD AI & Big Data algorithm engineer) and An Shan (Master's from Shandong University Robotics Research Center, senior algorithm engineer at JD AI & Big Data). Both specialize in large‑scale image retrieval and computer vision.

JD's panorama main‑image technology showcases 360° product views by automatically processing massive video assets. The pipeline includes video classification (clockwise, counter‑clockwise, static), frame extraction, cropping, and ordering, requiring an automated solution due to the volume of data.

The article explains optical flow as the pixel‑wise motion field between consecutive images. It traces the history from the 1950s to modern deep‑learning methods, describing the Horn‑Schunck variational model (brightness constancy and smoothness constraints) and the Lucas‑Kanade sparse method (linear equations solved by least squares).

Improvements such as coarse‑to‑fine pyramidal strategies, sparse vs. dense estimation, and robust handling of occlusion, noise, and illumination changes are discussed. The transition to deep learning is highlighted with FlowNet, which directly predicts optical flow from image pairs using CNNs. Two architectures are described: FlowNetSimple (FlowNetS) and FlowNetCorrelation (FlowNetC), along with their refinement stages and supervised training using synthetic datasets like Flying Chairs.

Evaluation metrics (average end‑point error, average angular error) and visualization techniques (color‑coded flow fields) are presented.

Application cases include:

Video object detection: using optical flow to exploit motion differences between foreground and background, and accelerating detection via methods like Deep Feature Flow.

Semantic segmentation: joint optimization of flow and segmentation (e.g., Object Flow).

Image synthesis: View synthesis via Appearance Flow (AppFlow), which learns a flow field to warp source pixels for novel viewpoints.

The article concludes that optical flow has progressed through three stages—early development, optimization‑driven advances, and deep‑learning breakthroughs—yet challenges remain in handling occlusion, fast motion, and limited ground‑truth data, motivating future semi‑supervised and unsupervised research.