Using etcd and jetcd for Master‑Standby Service Coordination in Java

etcd is a strongly consistent distributed key‑value store that offers reliable data storage for distributed systems, handling leader election and tolerating node failures. It is commonly used for distributed configuration, locks, service coordination, and registration, and is implemented in Go, providing a simple HTTP API.

The article presents a master‑standby service scenario where only one instance can be active at a time, such as a binlog parsing service that feeds data to Kafka and downstream systems. To achieve high availability, etcd is used to coordinate the active and standby instances.

jetcd Dependency

<dependency>
    <groupId>io.etcd</groupId>
    <artifactId>jetcd-core</artifactId>
    <version>0.3.0</version>
</dependency>

Client Initialization

Client client = Client.builder().endpoints(
        "http://127.0.0.1:2379",
        "http://127.0.0.1:3379",
        "http://127.0.0.1:4379"
).build();

Key APIs

Lock lock = client.getLockClient();
Lease lease = client.getLeaseClient();

Lease provides grant(long ttl) to create a lease that automatically deletes associated keys after ttl seconds, and keepAlive() to renew the lease.

Lock offers lock(ByteSequence name, long leaseId) and unlock(ByteSequence lockKey) to implement a distributed lock, where the lease ID defines the lock’s lifetime.

Using these APIs, a master‑standby switch can be realized by granting a lease, keeping it alive, and acquiring the lock. When the primary instance fails, its lease expires (as fast as 1 second), allowing a standby instance to obtain the lock and become active.

Sample Code

ByteSequence lockKey = ByteSequence.from("/root/lock", StandardCharsets.UTF_8);
Lock lock = client.getLockClient();
Lease lease = client.getLeaseClient();
long leaseId = lease.grant(lockTTl).get().getID();
lease.keepAlive(leaseId, new StreamObserver
() {
    @Override
    public void onNext(LeaseKeepAliveResponse value) {
        System.err.println("LeaseKeepAliveResponse value:" + value.getTTL());
    }
    @Override
    public void onError(Throwable t) { t.printStackTrace(); }
    @Override
    public void onCompleted() {}
});
lock.lock(lockKey, leaseId).get().getKey();

Complete Test Cases

public class JEtcdTest {
    private Client client;
    private Lock lock;
    private Lease lease;
    private long lockTTl = 1; // seconds
    private ByteSequence lockKey = ByteSequence.from("/root/lock", StandardCharsets.UTF_8);
    private ScheduledExecutorService scheduledThreadPool = Executors.newScheduledThreadPool(2);

    @Before
    public void setUp() {
        client = Client.builder().endpoints(
                "http://127.0.0.1:2379",
                "http://127.0.0.1:3379",
                "http://127.0.0.1:4379"
        ).build();
        lock = client.getLockClient();
        lease = client.getLeaseClient();
    }

    @Test
    public void lockTest1toMaster() throws InterruptedException, ExecutionException {
        long leaseId = lease.grant(lockTTl).get().getID();
        lease.keepAlive(leaseId, new StreamObserver
() { ... });
        lock.lock(lockKey, leaseId).get().getKey();
        scheduledThreadPool.submit(() -> {
            while (true) {
                System.err.println("我是主服务开始工作了");
                TimeUnit.SECONDS.sleep(1);
            }
        });
        TimeUnit.DAYS.sleep(1);
    }

    @Test
    public void lockTest2toStandby() throws InterruptedException, ExecutionException { /* similar to above, prints standby message */ }

    @Test
    public void lockTest3toStandby() throws InterruptedException, ExecutionException { /* similar to above, prints standby message */ }
}

Running the three tests simulates a one‑master‑two‑standby high‑availability architecture. Only one instance prints its working message at a time; when the active instance is stopped, a standby immediately takes over, demonstrating effective failover.