iOS Multi‑Device Remote Control and Automated Testing Platform

Background

iOS remote control technology has some industry solutions that allow an app to be installed and operated on a remote iPhone, which is useful for compatibility testing. However, existing solutions suffer from UI screenshot latency, device disconnections, and low stability, especially for desktop‑like gestures such as swipe, long‑press, and full‑screen video playback. Most services rely on a single Mac mounting a single iPhone, which is costly. In addition to functional testing, apps need long‑duration stability testing (Monkey testing) that repeatedly traverses UI pages and monitors crashes.

On Android, many open‑source tools (Appium, minicap+minitouch, Macaca, uiautomator) provide large‑scale device farms. For iOS, achieving parallel, long‑running UI traversal with low cost and high stability remains difficult.

Solution Overview

The iQIYI testing team built an iOS driver kernel and a real‑device resource pool to address these challenges.

iOS Driver Kernel

After evaluating several tools, the team chose Facebook’s open‑source iOS automation framework WebDriverAgent (WDA) as the driver engine because it supports multiple iOS devices on a single Mac‑mini and aligns with Appium’s underlying implementation.

Re‑implemented tap, long‑press, and drag operations using Apple’s native APIs.

Optimized the Home button handling to reduce latency.

Replaced the screenshot method with Apple’s XCUIScreen API, adding adaptive size storage to greatly lower delay.

Fixed full‑screen video playback issues in WDA.

Resolved disconnections caused by Xcode builds, achieving high‑stability remote control.

Addressed WDA heartbeat status limit and UI size loss after long‑run sessions.

Effect after Optimization

The UI desktop now responds within 150 ms for clicks and swipes, making remote operation smoother.

Device Resource Pool & Task Distribution

A pool of iPhone devices is managed by a Mac‑mini cluster (one Mac‑mini handling three iPhones). The system provides remote rental capabilities: user registration, time‑limited usage, app installation, remote desktop sync, and responsive UI interaction.

Task management backend is containerized and uses MQ messages to trigger workers on the Mac‑mini nodes. Logs are shipped via Logstash to a centralized logging platform. Monkey test screenshots are stored in object storage with expiration policies.

Workers handle device plug‑in/out detection, WDA compilation/startup, dynamic port allocation, and device release. Additional services include app installation, future Monkey task execution, and performance‑log collection.

Auxiliary Services

UI Detection Service : Captures a screenshot every second during Monkey execution and sends it to a UIChecker service. The service uses a YOLOv3‑based deep‑learning model (52 convolution layers + 1 fully‑connected layer) to detect UI anomalies such as garbled text, error dialogs, or black screens within 20‑50 ms per image.

Performance Log Service : Retrieves CPU, memory, and network usage of the tested app via Instruments (once per second). Trace files are parsed into readable formats and visualized on a log‑analysis platform. Instruments are periodically restarted to avoid data gaps.

Crash Monitoring Service : Uses idevicesyslog to filter app‑related syslog entries and idevicecrashreport to collect raw crash binaries. Symbolication is performed with symbolicatecrash and dSYM files to produce human‑readable crash reports.

All collected information (device info, UI screenshots, crash logs, performance curves) is aggregated into a bug‑management platform, automatically creating tickets for the responsible product teams.

Future Plans

• Resolve Xcode build failures caused by system dialogs. • Apply deep‑learning‑driven decision making to adjust Monkey traversal depth and breadth.