Design and Implementation of iQIYI's User Feedback Analysis System

Introduction

With the turning point of internet user growth, competition has shifted to battling for existing users. Delivering an excellent user experience becomes critical, and user feedback is the most direct expression of product experience. Feedback is massive and diverse, providing valuable signals for product optimization.

The initial handling of feedback faces four main challenges: (1) multiple channels and huge data volume make problem extraction costly; (2) feedback quality varies, increasing analysis cost; (3) long analysis chains lead to low closure rate and latency; (4) lack of effective evaluation methods causes delayed recall of specific issues.

Two implementation approaches exist: using external professional feedback services for trend monitoring, or building an in‑house feedback analysis system that emphasizes automatic classification, clustering, and alarm generation. iQIYI’s testing team chose the latter.

Solution Design

The solution aims to establish a generic service capability that improves recall efficiency of head‑line problems, quickly identifies issues, and assists business teams in solving them. The overall architecture links feedback mining, analysis & localization, repair closure, and effect tracking.

Feedback Ingestion

This stage performs data preprocessing, filters multi‑channel inputs, aligns data into a unified schema, and standardizes classification fields, delivering high‑quality data for downstream mining and analysis.

Feedback Mining

Three core capabilities are built:

1. Multi‑level Automatic Classification – Rule‑based first level for high precision, followed by fastText‑based second level (n‑gram features) and Word2vec similarity as a third‑level backup. This hierarchy raised classification accuracy by 40% and alarm accuracy by 30%.

2. Incremental Time‑Window Clustering – Single‑pass clustering processes streaming feedback in one pass, using TF‑IDF combined with Word2vec cosine similarity to improve recall. The clustering supports three scenarios: monitoring alarm hotspot extraction, duplicate‑issue detection, and real‑time small‑batch problem mining.

3. High‑Quality Feedback Identification – A multi‑dimensional quality assessment model evaluates factors such as scenario reasonableness, content consistency (image‑text, log‑text), and, for logged‑in users, historical feedback analysis. This model reduces follow‑up volume by about 80%.

Analysis & Localization

Based on classified feedback, the system narrows down problem scope using six dimensions (time, platform, version, region, carrier, and source aggregation). For high‑quality tail‑issue recall, configurable pipelines enable rapid localization and visual flow‑chart output.

Repair Closure

Standardized multi‑role flow, closure monitoring, and pre‑emptive FAQ generation improve closure rate and cycle. Features include one‑click reporting, automatic assignee routing, and post‑closure user notification.

Custom Issue Tracking

Users can define tracking tasks with dimensions such as category, keywords, platform, version, device, region, and carrier. Custom alerts detect regressions after initial resolution.

Process/Result Metrics

Metrics such as closure rate and closure cycle are collected to evaluate each stage and guide continuous improvement.

Overall Framework

The platform consists of an interactive layer (visual UI), a service layer (generic services via configurable modules), and a data layer (task scheduling and real‑time front‑end updates). Multiple business lines have adopted the platform, achieving significant gains in closure rate and reduced cycle time.

Conclusion

User feedback is a rich source of intelligence. The built system provides an end‑to‑end solution from problem discovery to repair closure, enhancing user experience and loyalty. Future work includes expanding feedback channels, automating duplicate issue association, and improving user outreach.