Full-Process Traceability Management for Machine Learning Models: Challenges, Methods, and Solutions

Abstract Artificial intelligence and machine learning are key technologies for extracting value from massive data, yet building and deploying ML models involves complex, iterative processes that lack systematic traceability, leading to high maintenance costs and reduced efficiency.

Background The rapid development of big data and AI has enabled enterprises to derive business insights, but prolonged model lifecycles and frequent updates make it difficult to record and manage experiments, configurations, and runtime environments.

1.1 Challenges in ML Process Management (1) Experiment design and model building involve many variables (data sources, feature engineering, algorithms, hyper‑parameters, hardware) with insufficient recording tools, making results hard to reproduce. (2) The end‑to‑end pipeline (data cleaning, feature engineering, training, evaluation, deployment) is long and tightly coupled, so upstream changes propagate complexly downstream. (3) Lack of unified version control across stages increases communication overhead and risks inconsistent outputs. (4) Model services often have multiple versions, and managing these versions, their dependencies, and rollback procedures is more complex than traditional code versioning.

1.2 Existing Methods and Their Limitations Git provides excellent code versioning but cannot capture non‑code artifacts such as data, environment parameters, and model files. Comet tracks experiments and visualizes comparisons but does not link experiment results to production services and lacks runtime parameter management. Huawei Cloud ModelArts offers full‑lifecycle visual management of data, training jobs, and inference services, yet it does not version code or handle multi‑version service dependencies.

2. Critical Issues to Solve To achieve full‑process traceability, the following technical problems must be addressed: (1) Versioned management of data cleaning, feature engineering, training, evaluation, and associated hardware/software parameters. (2) Versioned management of prediction pipelines, including data preprocessing, model inference, and runtime settings, and linking the selected model version to downstream services. (3) Traceability of online model services, monitoring of quality metrics, alerting, and automated or semi‑automated rollback and model selection based on defined thresholds.

3. Proposed Full‑Process Traceability Methods

3.1 Model Training Traceability Use Git for code versioning and DVC for data and model versioning. Store each training job’s configuration (data version, code version, hyper‑parameters, hardware specs, evaluation metrics) in a relational database, creating a record per job version. This enables comparison across versions and reproducibility.

3.2 Model Prediction Traceability Apply the same Git + DVC approach to the prediction pipeline (data cleaning, feature engineering, inference). Record pipeline versions, input data versions, model files, and runtime parameters in the database, linking each prediction job to its originating training job.

3.3 Online Service Traceability Deploy the prediction pipeline as an online service, monitor service health and quality metrics, and trigger alerts or circuit‑breaker actions when thresholds are breached. Enable automatic or manual selection of a backup model based on performance criteria, and record all version relationships for root‑cause analysis.

Conclusion By integrating traceability across training, prediction, and online service stages, the proposed framework allows precise tracking of model lineage, facilitates version comparison, supports automated rollback, and improves overall efficiency and quality of AI development platforms.

References 1. Sculley, D., et al. (2015). Hidden technical debt in machine learning systems. 2. https://git-scm.com 3. https://dvc.org 4. https://www.comet.ml 5. https://www.huaweicloud.com/product/modelarts.htm