Design and Implementation of the Diablo Distributed Configuration Management Platform

Application Scenarios of Distributed Configuration Platform

As business systems grow, configuration items such as data source settings, component parameters, and business options increase; while many configurations are stable and stored in files, some change frequently and need near‑real‑time updates, prompting the use of a distributed configuration management platform.

Distributed Configuration (Configuration Center): Common Scenarios and Requirements

Update certain configurations without restarting applications and have changes take effect almost instantly.

Manage configurations centrally instead of scattering them across individual application files.

Handle configurations that change frequently and follow a regular pattern.

Serve as a shared component for other distributed systems, similar to Zookeeper.

…

Basic Characteristics Required for a Distributed Configuration Platform

High availability: the server cluster must have no single point of failure; any surviving node should continue to provide service.

Fault tolerance: clients should keep running even when the configuration platform is temporarily unavailable.

High performance: fetching configurations must not introduce unacceptable latency.

Reliable storage: include backup, disaster recovery, and consistency, typically solved with databases and operational measures.

Near‑real‑time effect: clients must promptly perceive configuration changes.

Load balancing: distribute client requests evenly across server nodes to improve stability and performance.

Scalability: the cluster should support seamless expansion without service disruption.

…

Design and Implementation of the Diablo Distributed Configuration Platform

Diablo Architecture Design

The architecture is lightweight and consists of three main parts:

Apps: various business application instances.

Servers: a server cluster that provides configuration‑fetching services to clients.

Redis Storage Service: used for persisting configuration data.

Diablo Model Design

App: an abstract representation of an external system that uses the platform; it also carries security‑related settings such as signing keys and encryption keys.

Config: the core data item, containing a name, value, and metadata (e.g., MD5 hash) to detect changes.

Peer Server Cluster

All server nodes are peers with no master/slave relationship, eliminating issues like data‑sync latency or loss inherent in master‑slave architectures.

High Performance Handling

Clients read configurations from a local cache; when a configuration is modified via the console, the platform notifies clients to refresh their caches.

Using Redis for Storage

The platform stores data in Redis using simple keys (e.g., apps:1 ) and hash structures rather than JSON strings. It avoids special Redis commands such as pipelines or transactions to remain compatible with various Redis high‑availability solutions (Redis Cluster, Redis Proxy, etc.).

Client Implementation

Retry‑wait mechanism ensures that temporary server‑cluster outages do not affect client operation.

Load balancing uses consistent hashing to assign a specific server node to a client request, keeping subsequent requests on the same node.

Extra parameters (client version, language type, etc.) are sent alongside configuration data for compatibility and language‑specific handling.

Real-time Configuration Updates

Two modes are possible:

Pull: clients periodically poll the server; simple but introduces latency or load depending on the interval.

Push: server pushes updates to clients via long‑lived connections; more complex but provides true real‑time effect.

Diablo adopts a special Pull mode called Long Pulling (long‑polling). After a client sends an HTTP request, the server holds the request open until a configuration change occurs or a timeout expires, then responds, reducing request frequency while achieving near‑real‑time updates. Java long‑connections are implemented using the Servlet AsyncContext API.

Client Language Types

Currently Diablo provides a Java client; future support for Node, Go, Python, etc., is planned. Since Diablo exposes only HTTP APIs, any language that follows the API contract can implement its own client. The interaction diagram is shown below:

Practical Recommendations for Diablo

Server‑cluster deployment: use a reverse proxy such as Nginx to forward requests, allowing client applications to remain unchanged when the cluster topology changes.

Application environment separation: distinguish environments (dev, test, prod) by using different application names (e.g., app_dev, app_test). Production should be isolated and deployed independently.

Conclusion

The above details the design and implementation considerations of a lightweight distributed configuration management platform, illustrated with the open‑source project Diablo. Readers are encouraged to raise issues or fork the project on GitHub.