iQIYI's Automated App Recording and Playback System for Mobile UI Testing

When faced with a large number of app functional regression test cases, test development engineers often struggle with tight delivery schedules and lengthy regression cycles. To improve test quality and efficiency, iQIYI's technology team independently developed a new automated recording‑playback system for mobile apps.

System Overview

The system provides one‑click environment configuration, daily test development automation, rapid script generation, and full cloud‑based device handling, eliminating the need for local environment setup.

Resource Management Layer

This layer includes APP management, script management, scene (collection) management, task distribution, and test result management. It supports multiple app versions, links scripts to scenes, allows loop and order configuration, and plans to add shared variable visualization.

Task Distribution Process

After a scene is selected, the task is submitted to the server, which downloads script files from object storage, leases cloud devices, and dispatches tasks to Docker containers that maintain USB connections to the devices for stable UI automation.

Script Recording Tool

The web‑based IDE runs on iQIYI's mobile cloud resource pool and offers:

Frontend text generator that records UI actions and generates Python code.

Virtual terminal (ptmx) that receives the generated script and runs a Python interpreter for playback debugging.

Mobile APP client (iqiyi_uiautomator) that executes commands on the device.

Special Components

Mobile desktop video recording – captures MP4 videos to measure app launch time.

Online OCR element recognition – identifies text‑based UI elements and generates corresponding code.

Online image matching – uses SIFT feature matching for click actions on matched sub‑images and provides cloud‑stored screenshots.

Toast‑handling component – automatically dismisses system or app pop‑ups using OCR keyword detection.

App installation tool – supports manual and automatic installation by integrating with iQIYI's internal app packaging system.

Future Plans

The platform will extend to targeted Monkey testing, pingback automation, endpoint monitoring, and integration with CI/CD pipelines to enable test‑left‑shift, reducing cost and increasing efficiency. Additional improvements include streaming device screens via cloud ARM servers for lower latency interaction.