Automated Testing: From Data-Driven to AI-Powered - A Comprehensive Guide

Software testing is an indispensable part of the software development lifecycle and plays a crucial role in improving software quality. With the development of software testing, automated testing technology has also improved significantly.

This article first introduces the concept, classification, and current status of automated testing, and provides detailed analysis of the implementation principles of automated testing on different platforms. Through comparison of mainstream automated testing frameworks and tools, it points out the pain points and difficulties in implementing automated testing on different platforms.

Concept of Automated Testing: Automated testing is the process of converting human-driven testing behavior into machine execution.

Project Characteristics Suitable for Automated Testing: Projects with stable requirements, frequent regression testing needs, and complex test scenarios benefit most from automation.

Classification of Software Testing:

By Project Process: Unit testing, integration testing, system testing, regression testing, acceptance testing

By Technique: Black-box testing, white-box testing, gray-box testing

By Function: Logic function testing, UI testing, usability testing, installation testing, compatibility testing

By Performance: Time performance testing, space performance testing

By Automation: Functional automation, performance automation

Current Status of Automated Testing:

Unit Testing (70%): Tests the smallest testable units in software, written by developers using frameworks like XUnit

Interface Testing (20%): Tests interfaces between system components using tools like Jmeter and Postman

UI Testing (10%): Validates layout, style consistency, and UI functionality using Selenium and Robot Framework

PC UI Automation Testing: Each browser vendor provides a WebDriver that implements Selenium's WebDriver protocol, enabling browser control through HTTP requests.

Mobile UI Automation Testing:

Android: Based on UIAutomator and UIAutomator2, with earlier support from instrumentation framework. UIAutomator2 features include WiFi connectivity, real-time screen casting, precise device control, and the atxserver2 device management platform.

iOS: Based on UIAutomation and UITesting (introduced in Xcode 7). UITesting leverages iOS Accessibility features, supports OC and Swift, and includes built-in WebView testing capabilities.

Exploration: From Data-Driven to Keyword-Driven Testing:

The article explores three key abstractions:

Separation of UI element names and test object names: Map UI elements to logical objects, making tests independent of UI changes

Separation of test description from implementation details: Test descriptions specify what to test and expected results without depending on specific platforms

Separation of scripts and data: Extract test data from scripts for independent maintenance

Future Outlook - AI-Based Testing:

A complete mobile UI automation flow should include both functional and visual aspects. While functional automation is possible with mainstream frameworks, visual testing requires image recognition, image similarity matching, and audio matching technologies.

Following AI's success in games like Go, automated testing is evolving toward AI direction. Using deep learning, machines can make decisions through iterative training. Representative AI automated testing practices include iQIYI's Aion framework and Tencent Gaming QA's AI automated testing system.