Implementing Nearest Routing in Dubbo for Multi‑DataCenter Deployments

Problem Background

Vivo's internet services have grown rapidly, leading to large numbers of service instances. A single data center cannot host all machines, so multi‑data‑center or multi‑region deployments are required. In‑data‑center network latency is about 0.1 ms, intra‑city latency is ~1 ms, and inter‑city (e.g., Beijing‑Shanghai) latency exceeds 30 ms. Heavy cross‑data‑center calls increase request latency and degrade user experience.

Solution Overview

The goal is to keep RPC calls within the same data center or the same region whenever possible. The consumer should first call providers located in its own data center; if none exist, it falls back to same‑city or cross‑region providers. This strategy is called nearest routing . Vivo uses the Alibaba open‑source RPC framework Dubbo and extends it to provide a Dubbo‑specific nearest‑routing capability.

Technical Principle

When a provider registers, it includes its data‑center label (field app_loc ) in the registry. After enabling the nearest‑routing rule, the consumer filters the provider list to keep only those whose app_loc matches its own, thus achieving locality‑aware calls.

Implementation Details

The open‑source Dubbo does not provide nearest routing out of the box, so Vivo extends the Dubbo source code. The core extension implements the org.apache.dubbo.rpc.cluster.Router interface with a new router named NearestRouter . The following code snippet shows the routing logic:

public
List
> route(List
> invokers, URL consumerUrl, Invocation invocation) throws RpcException {
    // validate application name (this.url -> routerUrl)
    String applicationName = getProperty(APP_NAME, consumerUrl.getParameter(CommonConstants.APPLICATION_KEY, ""));
    boolean validAppFlag = application.equals(applicationName) || CommonConstants.ANY_VALUE.equals(application);
    if (!validAppFlag) {
        return invokers;
    }
    String local = getProperty(APP_LOC);
    if (invokers == null || invokers.size() == 0) {
        return invokers;
    }
    List
> result = new ArrayList
>();
    for (Invoker invoker: invokers) {
        String invokerLoc = getProperty(invoker, invocation, APP_LOC);
        if (local.equals(invokerLoc)) {
            result.add(invoker);
        }
    }
    if (result.size() > 0) {
        if (fallback){
            // 开启服务降级，available.ratio = 当前机房可用服务节点数量 ／ 集群可用服务节点数量
            int curAvailableRatio = (int) Math.floor(result.size() * 100.0d / invokers.size());
            if (curAvailableRatio <= availableRatio) {
                return invokers;
            }
        }
        return result;
    } else if (force) {
        LOGGER.warn("The route result is empty and force execute. consumerIp: " + NetUtils.getLocalHost()
                + ", service: " + consumerUrl.getServiceKey() + ", appLoc: " + local
                + ", routerName: " + this.getUrl().getParameterAndDecoded("name"));
        return result;
    } else {
        return invokers;
    }
}

The router can be configured via Vivo's internal service‑governance platform, which provides a visual UI for enabling/disabling the rule, setting fallback thresholds, and applying the rule per‑application.

Refined Plans and Future Work

While the current solution solves most immediate problems, several challenges remain:

The nearest‑routing code is bundled with Dubbo; upgrading Dubbo versions is required for any changes, leading to long upgrade cycles.

Service registration currently only carries data‑center information; richer metadata (e.g., rack, neighboring data‑centers) could enable more sophisticated routing.

Business traffic‑gray‑release strategies often need to combine nearest routing with Dubbo's conditional or tag routing.

When the nearest‑routing rule automatically fails, the business side lacks timely awareness.

Proposed directions include:

Adopting cloud‑native Service Mesh solutions to decouple complex routing logic from business code, leveraging Dubbo 3.0's upcoming cloud‑native support.

Integrating the registry with an internal CMDB to expose richer topology data.

Providing composite routing policies (nearest + conditional + tag) in the governance platform.

Building SDK‑side monitoring and alerting for routing failures and automatic adaptations.