Media Experience Quality Assessment: Visual Perception and Objective Quality Metrics

On October 15, the REDtech Youth Technology Salon featured Professor Zhai Guangtao from Shanghai Jiao Tong University, who presented a talk titled “Media Experience Quality Evaluation”. The talk introduced the human visual system and explained the significance and technical approaches for evaluating media experience quality.

The human eye receives visual information that undergoes complex processing in the retina, optic nerve, thalamus, and visual cortex (V1‑V4, MT). More than 50% of cortical neurons are involved in visual perception, and phenomena such as Troxler fading illustrate how the brain can ignore unchanging peripheral visual input.

Statistical data show the massive scale of image and video generation (over 1.5 trillion images captured in 2022, video traffic exceeding 80% of internet traffic). Most user‑generated content (UGC) is never viewed, and low‑quality media consumes significant storage and bandwidth.

Quality assessment can be categorized as full‑reference, reduced‑reference, or no‑reference. Full‑reference methods compare a distorted video with its original; reduced‑reference uses partial information; no‑reference evaluates quality using only the distorted media.

Perceptual signal processing, rooted in early work by D. Marr and later Nobel‑winning research by D. Hubel and T. Wiesel, involves three steps: (1) building a visual model that simulates perception, (2) designing evaluation algorithms to measure perceived quality, and (3) using the results to improve media quality.

Professor Zhai’s group proposes a structured visual perception model that combines physiological insights with information‑theoretic modeling. The model includes retinal filtering, local structure description, and a free‑energy perception component. An energy‑inversion technique estimates the original image’s free‑energy information from a distorted image, enabling high‑accuracy no‑reference quality assessment.

They also introduce a pseudo‑reference approach: instead of reconstructing the original, additional controlled distortions are added to the distorted image. The similarity between the pseudo‑reference and the distorted image indicates quality, offering a fast and scalable solution for large‑scale UGC video quality assessment (UGC‑VQA).

Audio‑visual perception is addressed by integrating audio features with visual quality metrics, either through correlation analysis or end‑to‑end deep learning models. An audio‑visual attention model and a large‑scale audio‑visual quality database have been built to support this research.

Subjective quality is commonly expressed as Mean Opinion Score (MOS), but relying solely on the mean can be misleading. The distribution of opinion scores (OSD) often exhibits long tails, skewness, or bimodality. To model these distributions, an α‑stable statistical model is employed, and parameters are estimated to predict perceived quality more accurately.

The presentation concluded with a Q&A session discussing challenges in UGC video quality (e.g., frame‑wise quality fluctuations) and the feasibility of using EEG‑based perceptual loss functions, highlighting current limitations due to sparse EEG sampling.