TiDB 2.1.x to 4.0.13 Upgrade and Data Migration Guide

Introduction The existing TiDB clusters run version 2.1.[5,7,8,17] and suffer from management difficulty, hotspot issues, plan invalidation, and performance bottlenecks. Direct in‑place upgrades would require multiple jumps (2.1 → 3.0 → 4.0) and risk irreversible failures, so a data‑migration approach is chosen.

Environment Overview The old cluster consists of 3 PD nodes (port 5017), 3 TiDB nodes (port 4000), 3 TiKV nodes (port 20117), and auxiliary services (Alertmanager, Prometheus, Grafana, VIP, DNS). The new cluster adds Pump and Drainer components and uses a different port scheme (e.g., PD 13002, TiDB 15002, TiKV 17002, Pump 23001, Drainer 24001). Tables describing roles, counts, and ports are provided in the source.

Upgrade Process The DBA follows ten major steps:

Print current client IP list from each TiDB node.

Confirm whether the application has a reconnection mechanism.

Enable binlog on the old cluster (requires rolling restart of TiDB).

Deploy the 4.0 environment and import full data.

Synchronize incremental data via Pump/Drainer.

Validate data consistency between old and new clusters.

Deliver the new environment (new DNS and ports, read‑only account).

Business verification.

Synchronize permission tables (old 2.1 → new 4.0).

Switch traffic.

Key Commands (wrapped in code tags)

ansible # /opt/soft/mysql57/bin/mysql -u root -h old.tdb1 -P 4000 -ppassword
mysql> SELECT DISTINCT host FROM information_schema.processlist;

# Edit inventory for Pump
[monitored_servers]
monitor-pump1 ansible_host=xxxx deploy_dir=/path/tidb-data/pump-23001
monitor-pump2 ansible_host=xxxx deploy_dir=/path/tidb-data/pump-23001
monitor-pump3 ansible_host=xxxx deploy_dir=/path/tidb-data/pump-23001

[pump_servers]
pump1 ansible_host=xxxx deploy_dir=/path/tidb-data/pump-23001
pump2 ansible_host=xxxx deploy_dir=/path/tidb-data/pump-23001
pump3 ansible_host=xxxx deploy_dir=/path/tidb-data/pump-23001

# Enable binlog in global vars
[all:vars]
pump_port = 23001
enable_binlog = True

# Deploy Pump and start it
ansible-playbook deploy.yml -l monitor-pump1,monitor-pump2,monitor-pump3,pump1,pump2,pump3 -i inventory.ini
ansible-playbook start.yml -l monitor-pump1,monitor-pump2,monitor-pump3,pump1,pump2,pump3 -i inventory.ini

Full Data Backup Use mydumper to export all tables from the old cluster, ensuring the operation runs during low‑traffic periods and adjusting gc and memory settings if needed.

Deploy New Cluster with TiUP Install TiUP, prepare a topology YAML (specifying PD, TiDB, TiKV, CDC, monitoring servers), verify SSH connectivity, then run:

# tiup cluster check tidb-oltp-xxx-v4.0.13 v4.0.13 topology-oltp-xxx.yaml
# tiup cluster deploy tidb-oltp-xxx-v4.0.13 v4.0.13 topology-oltp-xxx.yaml
# tiup cluster start tidb-oltp-xxx-v4.0.13

Incremental Sync Configure Drainer in drainer.yml , set drainer_port = 24001 , and deploy/start it with Ansible playbooks. Verify status via SHOW DRainer STATUS; or binlogctl for versions prior to 2.1.6.

Data Consistency Check Clone the check_data_for_mysql tool, adjust conf/check.conf for source/target hosts, and run the script inside a screen session. The tool compares schema and row counts, reporting success or mismatches.

Permission Synchronization A Bash script extracts users and privileges from the old cluster and generates CREATE USER and GRANT statements for the new cluster. After review, pipe the output to the new TiDB instance.

Traffic Switch Two strategies are described: a "brute‑force" approach where the business changes connection strings to the new DNS/port, and a "smooth" approach that updates the VIP's backend server list to point to the new TiDB nodes after draining the old ones. The smooth method minimizes write‑split and duplicate‑key issues.

Caveats - Versions prior to 2.1.6 report log_bin = 0 even when binlog is enabled; use binlogctl instead. - Ensure Pump is running before rolling restarts. - Verify that the tidb_binlog database is cleared before incremental sync if a full import failed. - After traffic switchover, keep the old DNS/VIP for a grace period to catch stray connections.

Conclusion The guide consolidates practical experience from 24 successful upgrades, offering a checklist and troubleshooting tips to help DBAs avoid common pitfalls when migrating TiDB clusters from 2.1 to 4.0.13.