Random Data Source Selection Issue When Adding Replicas in OceanBase 4.2.1.4 and Its Solutions

Background and Problem

A customer changed their disaster‑recovery architecture from a "two‑region two‑center three‑replica" setup to a "two‑region three‑center five‑replica" configuration. After the change, the majority‑vote could be formed within the same city, avoiding cross‑city network performance jitter. However, when adding a replica for a tenant (new replica placed in Shenzhen A data center), the data source was randomly selected—sometimes from Shenzhen, sometimes from Guangzhou. Pulling data from Guangzhou (which has only a 1 Gbps link) could fully occupy the bandwidth and cause congestion.

Issue Reproduction

The author reproduced the problem in a limited test environment by converting a "two‑region two‑center" setup to a "two‑region three‑center" setup. The steps included:

Viewing zone information via select svr_ip,a.zone,name,info from __all_zone a, __all_server b where a.name in ('region','idc','status','info') and a.zone=b.zone;

Checking tenant role information with select b.tenant_name,a.tenant_id,a.ls_id,a.zone,a.svr_ip,a.role from cdb_ob_table_locations a join __all_tenant b on a.tenant_id = b.tenant_id where b.tenant_name = 'mysql_ob' group by svr_ip,role order by zone;

Adding a new zone (IP 10.186.64.62, region "Shenzhen Nanshan") and verifying the zone list.

Adding a replica for the mysql_ob tenant and observing the replica‑level disaster‑recovery tasks, which showed the source replica IP as 10.186.64.10 (a follower node) even though the new replica was in the same data center as the leader.

Deleting the added replica, adjusting the tenant's primary_zone to "zone2;zone1", confirming leader switch to 10.186.64.10 , and attempting to add the replica again, which still selected the follower node as the data source.

The observation confirmed that the data source selection during replica addition is random and not tightly bound to the tenant's role.

Summary

In OceanBase 4.2.1.4, adding a tenant replica may randomly choose a data source, potentially leading to a "far‑away" selection that overloads cross‑city bandwidth.

Solution

Method 1 (Recommended) : Set the system parameter choose_migration_source_policy='idc' to prioritize migration sources within the same IDC. This parameter is available from OceanBase 4.2.1 BP7 onward.

ALTER SYSTEM SET choose_migration_source_policy='idc';

Method 2 (Recommended) : Manually specify the data source when adding a replica, e.g., ALTER SYSTEM ADD REPLICA LS XXX; . This requires OceanBase 4.2.1 BP8 or later.

// specify data_source
ALTER SYSTEM ADD REPLICA LS XXX;

Method 3 (Not Recommended, Temporary Workaround) : In environments where the original architecture was "two‑region two‑center three‑replica", temporarily stop the OBServer service in the Guangzhou data center. The remaining Shenzhen nodes still form a majority and can serve traffic, ensuring new replicas pull data only from Shenzhen, though this approach carries operational risk.