Hydra: One-Device-Multi-Control Solution for Mobile UI Compatibility Testing at Baidu

Introduction: Despite the rapid advancement of automation testing technology, manual testing remains a critical method for mobile quality assurance due to challenges in automated case construction costs and execution stability. Traditional manual testing requires human operators to execute test cases, with efficiency improvements relying on testers' operational proficiency. This article introduces Baidu's UI compatibility testing status and presents the "one-device-multi-control" concept as the foundation for the Hydra tool.

Background - Mobile UI Compatibility Testing: Mobile UI compatibility testing verifies UI interface consistency across different device models, resolutions, and screen sizes. In Baidu's application development process, traditional manual compatibility testing occurs in two main phases: (a) Function testing phase: typically 1-2 people testing on 1-3 devices for 10-20 hours; (b) Pre-launch full-function (regression) testing: 1-4 people testing on 5-12 devices for 20-50 hours.

Problems Faced: (a) Efficiency improvement is difficult - for example, testing 100 cases across 10 devices takes approximately 17 hours; (b) Insufficient compatibility coverage due to limited device availability across business lines.

Hydra Solution: Hydra addresses these issues through the "one-device-multi-control" approach, where a tester controls a "master" device while their operations are simultaneously replicated to multiple "slave" devices for UI validation. The tool connects to cloud device platforms to overcome physical device limitations.

User Requirements: Testers expect: (a) Accuracy - operations on master device must accurately replicate to slave devices; (b) Multi-device support - multiple devices, device types, and application types; (c) Usability - convenient, fast operation matching user habits; (d) Speed - fast control response.

Technical Architecture: Hydra uses B/S architecture with HTTP/WebSocket protocols. The core components include the Group Control Engine and Image Compositor, responsible for input and output of the "one-device-multi-control" functionality.

3.1 Group Control Engine: The design goal is to complete one action and execute multiple times. Challenges include: (a) Accurate coordinate mapping across different resolutions; (b) Execution timing across devices with different performance; (c) Timing issues for combined actions (e.g., click vs. long press). Solutions include using multi-scene high-performance image algorithms, waiting for all coordinate mapping to complete before execution, and maintaining action execution queues with timestamps.

3.2 Real-time Image Stream: This component transmits device images to the display. Challenges include: varying frame rates across devices, unstable network device frame rates, and frontend callback explosion. Solutions include limiting input frame rate to 16fps, reducing image dimensions, and using custom data protocols to composite multi-device images.

3.3 Multi-scene High-performance Image Algorithm: The coordinate mapping algorithm must balance performance, accuracy, and generality. The solution uses SIFT algorithm optimized through: (a) Region cropping to reduce computation; (b) Using two feature points to calculate target point relationships (offset angle and distance); (c) For list pages, using CNN for page recognition and DNN for object detection. Results achieve average coordinate mapping within 160ms with 97.52% accuracy.

3.4 Consistency Repair: To handle device image inconsistencies, Hydra supports individual device control during group operations and provides semi-automatic slide consistency repair using weighted feature point analysis.

3.5 Mobile Handheld Control: Hydra offers two handheld control schemes: (a) Remote control scheme using browser technology; (b) Android client solution using dual-layer floating windows to capture user input.

Results: Hydra has been deployed across multiple Baidu business lines, achieving 20%-70% weekly efficiency improvement in app regression, advertising testing, and operational activities.