Implementation Principles and Architecture of the Diamond Configuration Management System

Diamond is an internal Taobao system for managing persistent configuration data. It provides a simple, reliable, and easy‑to‑use service that allows applications to fetch configuration at startup and to sense configuration changes during runtime, with data persisted to both disk and database.

Key Features : simplicity (minimal structure reduces errors), reliability (no major failures after a year of production), and usability (client code requires only two lines and exposes straightforward APIs).

Persistence Mechanism : Subscribers read configuration directly from server‑side local files to minimize database load, using an API to fetch the latest data from the database when low‑latency is required.

Disaster‑Recovery Mechanism : Configuration is stored in multiple locations—database, server disk, client cache, and a manual‑intervention disaster directory. Clients query these sources in a fixed order (disaster directory → server disk → client cache) to ensure service continuity even when primary sources fail.

Architecture Overview (see image): The system consists of a server side that stores configuration and a client side that subscribes to updates via a singleton DiamondSubscriber created by DiamondClientFactory . The client registers a ManagerListener to receive configuration changes.

Client Subscription Example :

public class DiamondTestClient {
    public static DiamondManager manager;
    public static void main(String[] str) {
        initDiamondManager();
    }
    private static void initDiamondManager() {
        manager = new DefaultDiamondManager("group_test", "dataId_test", new ManagerListener() {
            public void receiveConfigInfo(String configInfo) {
                System.out.println("configInfo=" + configInfo);
            }
        });
    }
}

The DefaultDiamondManager constructor registers the listener, adds the dataId and group to the subscriber, and starts the subscriber thread.

Subscriber Startup :

public synchronized void start() {
    if (isRun) return;
    if (scheduledExecutor == null || scheduledExecutor.isTerminated()) {
        scheduledExecutor = Executors.newSingleThreadScheduledExecutor();
    }
    // initialize processors, http client, polling interval, etc.
    isRun = true;
    rotateCheckConfigInfo();
    addShutdownHook();
}

The rotateCheckConfigInfo method schedules a periodic task (default 60 seconds) that invokes three core methods: checkLocalConfigInfo , checkDiamondServerConfigInfo , and checkSnapshot .

Local Cache Check iterates over cached CacheData entries, reads local files, compares version numbers, and if a change is detected calls popConfigInfo to notify listeners.

Remote Server Check obtains updated dataId‑group pairs from the server, retrieves the latest configuration for each pair, and forwards it to the registered listeners.

Notification Logic :

void popConfigInfo(final CacheData cacheData, final String configInfo) {
    ConfigureInfomation ci = new ConfigureInfomation();
    ci.setConfigureInfomation(configInfo);
    ci.setDataId(cacheData.getDataId());
    ci.setGroup(cacheData.getGroup());
    if (subscriberListener.getExecutor() != null) {
        subscriberListener.getExecutor().execute(() -> {
            subscriberListener.receiveConfigInfo(ci);
            saveSnapshot(cacheData.getDataId(), cacheData.getGroup(), configInfo);
        });
    } else {
        subscriberListener.receiveConfigInfo(ci);
        saveSnapshot(cacheData.getDataId(), cacheData.getGroup(), configInfo);
    }
}

The system maintains a ConcurrentMap of listeners keyed by dataId+group , ensuring that all registered listeners receive updates.

Interaction Sequence Diagram (see image) illustrates the flow between client and server during subscription, polling, change detection, and listener notification.

Overall, Diamond demonstrates a robust design for distributed configuration management, combining simple APIs with comprehensive failover and polling mechanisms to keep large‑scale services consistently configured.