Automated Machine Learning System Architecture and Hyper‑Parameter Optimization Process

Machine‑learning techniques are increasingly applied in finance, advertising, recommendation systems and user‑behavior analysis, creating huge business value, but constructing high‑performance models remains time‑consuming because it requires selecting algorithms and tuning many hyper‑parameters for each specific dataset and task.

Two conventional approaches are used today: (1) expert‑driven model selection and manual hyper‑parameter tuning, which heavily depends on experienced algorithm engineers and cannot meet the rapid, large‑scale demands of modern enterprises; (2) using open‑source hyper‑parameter search toolkits, which still involve cumbersome definition, management and reuse of search spaces and experiment workflows, leading to low efficiency.

The proposed solution is an integrated automated‑ML system that supports two usage modes: a collaborative mode where domain experts interact with the system to refine search spaces and initialize tasks, and a “code‑free” mode where users configure tasks through a visual interface without writing code, both aiming to improve model quality and reduce manual effort.

The system consists of the following functional modules, denoted A–K: A → Git repository for version‑controlled model code; B → Container image repository providing runtime environments; C → Automated‑ML task designer (visual configuration); D → Hyper‑parameter search‑space management (definition, editing, visualization); E → Hyper‑parameter optimization engine; F → Optimization‑algorithm repository (default and user‑defined algorithms); G → Model training/evaluation engine (K8s‑orchestrated); H → Model repository (storage and management of optimal models); I → Task monitoring panel (real‑time metrics); J → Big‑data cluster for training/validation datasets; K → Model inference platform for deployment.

The core workflow proceeds as follows: (1) model developers commit code to the Git repository and define the hyper‑parameter search space in the task designer; (2) the backend launches the optimization engine, which samples hyper‑parameter combinations and dispatches training jobs to the training/evaluation engine; (3) the engines exchange parameters and performance feedback, iterating until the termination conditions set in the designer are met; (4) the best‑performing hyper‑parameter sets and trained models are stored in the search‑space management module and model repository, respectively, while the monitoring panel visualizes the entire process.

In summary, the paper introduces a complete AutoML architecture that abstracts and unifies hyper‑parameter search, model iteration, and surrounding infrastructure, offers expert‑assisted and no‑code pathways, and thereby reduces dependence on scarce ML expertise while accelerating the delivery of high‑quality models.