Statistical Process Control for Software Lifecycle: Building a Digital Data Warehouse and Applying Control Charts

This article explains how to use a software lifecycle tool integration platform to collect and analyze software process data, enabling real‑time monitoring and statistical analysis of project activities.

Software project management aims to balance cost, schedule, and quality, and full‑lifecycle process control is essential to achieve these goals.

Statistical Process Control (SPC) applies mathematical statistics and control charts to detect abnormal trends and loss of control in software development, providing early warnings and improving quality and productivity.

The key steps are:

Collect software process data from various engineering tools (e.g., requirements, defects, change sets, work orders).

Identify critical quality attributes and extract/transform the relevant data.

Select appropriate control charts based on data distribution characteristics.

Build a digital data warehouse to store unified process data for later analysis.

Extract and transform data from the warehouse to feed SPC models.

Choose SPC models suitable for the software industry.

1. Build a Digital Data Warehouse

During each development phase, tools such as Doors NG for requirements, JIRA or IBM RTC for planning and work items, and HP ALM for test and defect management generate digital records. Consistent use of these tools creates a rich source of process data that can be extracted and analyzed.

2. Data Extraction and Transformation

By integrating multiple tools, the platform can generate cross‑tool tables that capture the data of interest. When an artifact changes, a corresponding record is inserted into the database, enabling unified analysis.

For example, defects managed in JIRA, IBM RTC, HP ALM, or IBM CQ can be imported into a MySQL table, then extracted, transformed, and re‑loaded for reporting.

3. Choose Statistical Process Control Models

Histogram (frequency histogram) is a common SPC tool that visualizes data distribution by grouping process data on the horizontal axis and counting occurrences on the vertical axis, helping identify areas needing improvement.

Defect Lifecycle Duration Frequency Histogram

Using a stored procedure, the defect lifecycle durations are grouped and counted, producing a histogram that shows the distribution of defect fixing times, aiding efficiency analysis and prediction.

Defect Count by Status and Lifecycle Duration

This chart classifies defects by current status and the time spent in each status, measured in days, providing insight into defect distribution across states.

By aggregating cross‑tool data and applying real‑time SPC charts, software project management efficiency is improved, offering a new method to address common project challenges.