Advances in Image and Video Enhancement, Quality Assessment, and Multimodal AI Techniques

The presentation begins with an overview of real‑world image quality issues—spatial artifacts, temporal distortions, and color deficiencies—and summarizes recent progress in addressing them.

Spatial enhancement includes super‑resolution (pairwise supervised training, CNN → GAN → Transformer models, realistic degradation pipelines such as BSRGAN/REAL‑ESRGAN), de‑compression (real‑basic VSR data construction), and portrait enhancement (face‑focused GAN priors like GPEN).

Temporal enhancement covers de‑interlacing using UNet with multi‑frame frequency features, video stabilization (DUT network handling both global and local shake), and frame interpolation (optical‑flow‑based methods improved with transformers to handle large motion, repetitive textures, and occlusions).

Color enhancement discusses automatic color grading (pixel‑wise CNN to 3D‑LUT mapping, white‑box control of saturation, brightness, temperature) and colorization (pairwise training, recent Transformer‑based models such as ColorFormer and DDColor).

The importance of quality assessment is highlighted: MOS regression without references, component‑wise scoring (sharpness, compression, noise, colorfulness), and multi‑distortion ranking approaches that avoid direct scalar mapping. Techniques like CenseoQoE and clip‑based MOS prediction are described.

Multimodal opportunities are explored, including diffusion‑based super‑resolution (LDM‑V2 with text prompts) and large text‑image models for quality assessment, noting efficiency challenges and the need to remove textual conditioning.

Future work aims to leverage generative large models and multimodal AI for unified quality analysis and enhancement, improve evaluation efficiency, integrate one‑click analysis‑to‑processing pipelines, and pursue model lightweighting for mobile deployment.

The article concludes with a Q&A session addressing differences between image and video enhancement, training strategies for VQA models, on‑device deployment considerations, flicker issues, HDR handling, and the feasibility of large‑model approaches.