Noah: A Test Environment Governance Platform for Efficient Development and Testing

1. What is Noah

Noah is a test environment governance platform designed with infrastructure‑as‑code principles, allowing users to define environment templates and quickly build and deliver required test environments, thereby reducing the pain of manual DEV/QA environment setup.

2. Background

In 2014 Qunar faced rapid growth with over 5,000 daily deployments; building and maintaining test environments was costly and slowed product iteration. Key pain points were complex system dependencies, lengthy approval processes, and high maintenance costs.

3. How Noah solves environment challenges

Noah defines an environment as a collection of proxy, application, middleware, cache, and database. It offers one‑click creation, immediate teardown, rapid delivery via templated configurations, resource pooling, parallel construction, and environment self‑governance (auto‑cleanup, VM drift handling, fault isolation).

4. Environment build workflow

Noah abstracts the build process as a workflow with four phases—resource lock, delivery, network initialization, and validation—implemented as a DAG.

5. System architecture

The core orchestration engine, inspired by Kubernetes CRDs, treats each environment as a task and drives component delivery via a state machine.

6. Core feature – Soft Routing

Soft routing reuses application services across environments by adding an environment‑id to requests, enabling service‑level virtualization and reducing resource consumption. The routing flow involves recording the ID on the host, reporting by services, binding by clients, propagation via Qtrace, and routing decisions based on the ID.

7. Core feature – Baseline Library

A baseline MySQL library automatically creates test databases, syncs schema changes, and can execute custom SQL scripts, with backups stored on low‑cost MFS storage.

8. Database containerization

Containerizing MySQL saves ~80 % of machine resources, reduces provisioning time from minutes to seconds, and eases bandwidth pressure. Scheduling uses Consul agents to report host load and selects the highest‑scoring host for container launch.

9. KVM resource pooling

KVM pools pre‑create VMs to trade space for time, with a two‑level scheduling: OpenStack selects hosts based on tags and weight, then Noah distributes environment VMs across hosts using a round‑robin‑like algorithm to improve fault tolerance.

10. Middleware management

Noah provides fast, flexible installation of Tomcat, Elasticsearch, MQ, JDK, etc., via Salt for configuration management.

11. Future directions

Automated analysis and self‑healing of common build issues.

Support for more test scenarios such as unit, API, and performance testing.

Improved observability to narrow down and locate environment problems.