Building an AI-Powered Testing Ecosystem: From Core Principles to Automation

1. Test Capability Beyond Business

To focus on the essence of testing, the article first explains why the team temporarily sets aside specific business scenarios. The data platform testing group at Gaode deals with diverse business forms, yet the core goal of testing is to solve business problems. Over years, they have built a "one‑force for ten abilities" universal testing capability, continuously refined as AI permeates testing.

2. Clarify Test Role

The basic responsibilities of a test role are quality assurance, efficiency improvement, and process control. Cost is the inevitable product of efficiency; otherwise it is false efficiency.

Testing Team Development Stages

Stage 1 – Early Construction (Tool‑centric)

In the early stage, the team ensures the system does not go wrong online (core mission of delivery). Quality assurance (~80%) and process control (~20%) dominate. Tools are introduced to magnify potential issues, emphasizing standardised, repeatable test processes.

Stage 2 – Feature Iteration (Automation)

As the product stabilises, functional iteration creates a 1×N regression workload. Automation, driven by programming, becomes the main efficiency solution.

Stage 3 – Long‑term Consolidation (Platform‑centric)

Separate from single business scenarios, evolving into a testing platform.

Support development flow, enabling left‑shift in DevOps.

Extend from pre‑release testing to online monitoring (right‑shift).

This stage focuses on platform‑based value enhancement.

3. Test Ecology Before AI

The team explains daily work: OKR‑driven task breakdown, fragmented tools (Aone, Mayflower, Teambition, Kelude) causing information silos and context‑switch costs. A visual diagram illustrates these fragmentation issues.

From a service‑side perspective, testing must cover functional, performance, data, security, environment/configuration, and other aspects, requiring extensive tooling and platform support.

Ecology‑Based Problem Solving

Information aggregation for decision‑making (Agile delivery).

Test task modularisation (Agile testing).

Resource change tracking for risk assessment.

These three directions illustrate how the ecosystem integrates AI later.

4. AI Integration Points

Potential AI insertion points include content generation, code or unit‑test generation, and model‑based judgment. However, a systematic understanding of large models is still lacking.

Key AI use cases identified:

Content generation (structured data and narrative).

Information consulting (professional knowledge and decision support).

Information recognition (multimodal OCR and feature tagging).

AI’s value lies in its understanding ability, not just execution.

5. Finding the Right Points for AI

AI can assist in content generation, requirement analysis, test planning, risk prediction, and more, by leveraging a well‑structured knowledge base.

6. Building the Knowledge Base (Data Domain)

The knowledge base consists of multiple interconnected domains:

Project‑management domain (Aone, Yuque, DingTalk, emails).

AI perspective on project context.

Aone application domain (code, middleware, release data, test artefacts).

AI perspective on technical portrait.

Test‑ecosystem domain (automation scripts, expert knowledge).

AI perspective on testing methods.

Data domain (logs, user behaviour, monitoring, dashboards).

AI perspective on quantitative analysis.

Operations domain (quality‑operation processes, feedback loops).

AI perspective on operational impact.

These domains are linked, forming a knowledge graph that fuels AI understanding.

7. Knowledge Base Construction and Iteration

To keep the knowledge base up‑to‑date, the team proposes automatic iteration: code change detection triggers knowledge‑base updates, branch‑level file mapping ensures consistency, and scheduled jobs refresh the repository daily.

8. Requirement Enhancement

Requirement analysis is the starting point for intelligent testing. By enriching raw requirements with project context, keyword explanations, dependency mapping, and engineering details, the team creates a “big requirement” that AI can consume effectively.

Precise requirement parsing.

Test scenario prediction.

Risk prediction.

Automation support for test case generation.

9. Macro Plan for Requirement Enhancement

Three layers are defined:

Business‑mapping layer (select appropriate knowledge‑template).

Intelligent parsing layer (agents for keyword, scenario, data‑spec enhancement).

Link‑control layer (knowledge‑graph retrieval + agent recommendation).

10. From Requirement to Test Cases

First Act – Requirement Analysis

Structured analysis extracts functional items and details, forming a test skeleton.

Second Act – Test Case Generation

Structure‑based automatic extraction of test elements.

Expert knowledge injection (industry patterns, historical defects).

Platform‑specific operation steps insertion.

Third Act – Test Case Review

AI‑assisted review parses requirements, checks coverage, recommends missing cases, and links changes to test updates, creating a closed‑loop, high‑quality test suite.

11. Automated Test Cases

Automated cases combine natural‑language description with embedded resources (HTTP, HSF, DB, MetaQ, schedule) and data factories. The workflow splits description, maps resources, and generates executable steps, with agents handling context transformation and validation.

12. End‑to‑End AI‑Driven Testing Vision

The article concludes that when AI permeates every testing stage—from data aggregation, knowledge‑base construction, requirement enhancement, to automated execution—quality assurance becomes a collaborative dance between human expertise and machine intelligence.