Mastering Continuous Testing: A Guide to DevOps Maturity and Best Practices

Continuous Testing Overview

Continuous testing is an automated testing process that can be performed at any time within a software delivery pipeline, featuring left‑shift, right‑shift, and automation.

Maturity Model

The "DevOps Capability Maturity Model – Part 11: Continuous Testing" defines five maturity levels (1‑5) covering dimensions such as standards, left‑shift, right‑shift, automation, metrics, defect prevention, precise testing, decision support, intelligence, and continuous improvement.

Process Phases

The continuous testing workflow is divided into six stages: Requirements, Development, Integration, Acceptance & Release, Operations, and Continuous Feedback. Each stage describes the testing activities and their role in creating an end‑to‑end testing loop.

1. Requirements Stage

Testers participate in requirement reviews, test iteration planning, and test design, ensuring testability and driving test‑left shift.

2. Development Stage

Testing includes test case design, code quality management, unit testing, and smoke testing, supporting continuous testing‑driven development (CTDD).

3. Integration Stage

Automated builds and continuous integration enable integration testing, non‑functional testing, and regression testing before release.

4. Acceptance & Release Stage

Testing covers acceptance testing and low‑risk release testing to verify functional and technical compliance before production.

5. Operations Stage

Post‑release testing (TestOps) involves continuous monitoring and online testing to validate stability and security.

6. Continuous Feedback

Feedback includes test entry/exit criteria, reporting, defect management, quality and incident analysis, and development optimization.

Supporting Capabilities

In addition to the workflow, continuous testing requires general testing foundations (process standards, personnel, assets, environment, data, platform) and performance metrics that measure effectiveness across the entire DevOps pipeline.

Case Studies

In December 2023, the GOLF+IT Governance Forum announced that Agricultural Bank of China and Postal Savings Bank of China were the first state‑owned banks to pass the Level‑3 continuous testing assessment. Later, China International Capital Corp and Shenwan Hongyuan became the first securities firms to achieve the same.

These banks reported improvements such as higher automation coverage, a 99% reduction in technical debt, shortened release cycles, and enhanced data‑driven decision making.

2024 Continuous Testing Capability Evaluation

The first batch of 2024 Continuous Testing (CT) capability maturity assessments is now open for registration, running from March to June 2024, with results to be announced at the mid‑year conference.