Why Quality Is Low: Insights and Metrics from the 2019 Scaled Agile Spring Summit

The author participated in the 2019 Scaled Agile Spring Summit as a Topic Leader, focusing on the seventh of SAFe's 13 most common problems: "Quality is low" (质量低). The discussion began with a brainstorming session where each participant contributed over ten quality‑related sticky notes.

The 13 SAFe problems listed were: 1) Agile doesn’t fit large regulated solutions, 2) DevOps and Continuous Delivery are impossible, 3) Budgetary process inhibits agility, 4) Problems discovered too late, 5) No way to improve systematically, 6) Leadership style and culture clash with agility, 7) Quality is low, 8) Late delivery, 9) Inability to release value when needed, 10) Under‑estimated dependencies, 11) Growing technical debt, 12) Insufficient team‑level agile processes, 13) Poor morale.

Four reasons were given for selecting "Quality is low" as the focal point: its brevity and clarity, its depth for discussion, its connection to many other problems, and a personal intuition to explore a less‑familiar topic.

The article then examines what quality means. Quality is defined as the absence of undesirable attributes such as missing functionality, poor performance, or low usability. It distinguishes two perspectives:

Internal quality – process indicators like defect count, code coverage, unit‑test coverage, etc.

External quality – outcome indicators that reflect whether the product meets functional and non‑functional user needs (reliability, usability, security, performance).

External quality is the visible manifestation of internal quality; when internal safeguards hinder external outcomes, a misalignment occurs.

Quoting Peter Drucker, the article stresses that without measurement there is no management. Common measurement pitfalls include over‑emphasizing process metrics, focusing on local rather than global metrics, and concentrating on individual rather than system‑wide metrics.

Images illustrate the distinction between internal and external quality and the flow from global outcome metrics to local process metrics.

The author presents a three‑stage model (3S): "Smart" – deciding what to do, "Strong" – how to do it efficiently, and "Sensible" – verifying that the right thing was done. This model aligns with the Build‑In‑Quality principle, where quality is embedded throughout the software lifecycle rather than being a separate testing phase.

Practical tools are discussed, emphasizing that many agile tools are not software but methods and skills (e.g., physical Kanban boards). References to Mary Poppendieck’s "Lean Software Development – An Agile Toolkit" underscore the importance of non‑software tools.

Finally, the author’s bio is provided: Yao Dong, senior DevOps and Lean/Agile expert, former IBM DevOps technical director for Greater China, currently Chief Technology Evangelist for Huawei Cloud Software Development Cloud.