Design and Implementation of an Automated Test Environment Deployment Platform

1 Background

To support fast business growth and orderly iteration, product, development, and QA teams must continuously iterate and optimize features, verify code quality, and meet release standards. Test environment availability and stability are fundamental for development efficiency. As the New Oriental APP team expanded from a few dozen to hundreds of engineers and from a single test environment to multiple parallel environments, problems such as environment overlap, delayed updates, long setup times, cross‑environment interference, and data inconsistency became evident.

2 Challenges

2.1 Diverse Development Stacks

The platform must support a wide range of technologies, initially including PHP, Java, Go, and H5.

2.2 Inconsistent Frameworks and SDK Versions

Even within the same language, different frameworks and SDK versions are used (e.g., PHP 5.0 vs 7.0, Java Spring Boot vs legacy SSH+Tomcat).

2.3 Varied Middleware

Projects rely on many middlewares: RabbitMQ, Kafka, MySQL, TiDB, Oracle, SQL Server, etc.

2.4 Non‑standard Configuration Management

Service configurations are scattered: some are hard‑coded, others use different configuration centers such as XCloud or Apollo, making standardization difficult.

3 System Implementation

3.1 Architecture

3.2 Deployment Process

3.3 Deployment Steps

Users only need to perform two steps:

1. Enter an environment tag (identifying a set of resources) and click Save.

2. Select the project and branch to deploy, click Save, then click Deploy.

The system automatically creates Apollo configurations (when required), builds packages, creates Docker images, and performs Kubernetes deployment. After a successful deployment, health‑check URLs, IP/port, and container status are displayed. Domain names are generated dynamically from the environment tag, eliminating the need for manual domain registration.

3.4 Common Issues and Solutions

3.4.1 Docker‑Ready Projects

Most projects lacked Docker deployment configurations.

Solution: Add Dockerfile and related configuration files to standardize image creation and service setup.

3.4.2 Configuration Replacement

Project configuration files varied widely, making automated replacement difficult.

Solution: Standardize naming conventions for new projects and implement flexible matching functions for existing configurations.

3.4.3 Domain Resolution

Multiple environments and projects had inconsistent domain names.

Solution: Use wildcard domains and automatically generate Nginx configurations after deployment.

3.4.4 XCloud Apollo Integration

The platform’s Apollo A differs from the production Apollo B and requires admin privileges.

Solution: Add configuration in project files to point to Apollo A for the test environment, with default cluster settings stored in XCloud.

3.4.5 Redis Service Access

Redis instances in each environment had different IPs, and sentinel configurations referenced wrong IPs.

Solution: Use Docker exec to run a command that updates the sentinel configuration with the correct IP.

4 Current Usage

The system is now used by APP and content‑tool development and QA teams for daily development and testing environments, and it has been piloted in a small classroom setting.

5 Future Plans

Beyond promotion and feature enhancements, future work will focus on improving deployment performance and efficiency, monitoring test‑environment services, integrating with production deployments, and enhancing the self‑service experience for users.

6 Acknowledgements

Thanks to the operations team (Cao Lei), the Information Security team, architecture team, and the APP and content‑tool development and QA teams for their support.

Source: New Oriental APP QA Team