Using Alibaba Canal for Incremental Data Synchronization Between MySQL and Heterogeneous Data Stores

In large applications data often resides in heterogeneous stores (MySQL, Redis, Elasticsearch, etc.), and keeping them consistent requires incremental synchronization; Alibaba's open‑source tool Canal fulfills this need.

Canal is a MySQL binlog‑based incremental subscription and consumption component that can be used for database mirroring, data heterogeneity, indexing, cache updates, and more.

It works by mimicking a MySQL slave: it sends a dump request to the master, receives the binlog stream, parses it, and then performs user‑defined actions such as syncing to other databases.

Configuration example:

log-bin=mysql-bin          # enable binary log
binlog-format=ROW        # use row mode
server-id=1              # master server ID (unique)

MySQL replication offers three modes—Statement, Row, and Mixed—with Row mode recommended for Canal.

Create a dedicated Canal user with the necessary privileges:

CREATE USER 'canal' IDENTIFIED BY 'canal';
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
FLUSH PRIVILEGES;

After extracting the instance.properties file from the Canal distribution, set the master address, username, and password:

# Database address
canal.instance.master.address=MySQL_IP:Port
# Username / password
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
canal.instance.connectionCharset=UTF-8

Include the Canal client dependencies in your Maven project:

<dependency>
    <groupId>com.alibaba.otter</groupId>
    <artifactId>canal.client</artifactId>
    <version>1.1.6</version>
</dependency>
<dependency>
    <groupId>com.alibaba.otter</groupId>
    <artifactId>canal.protocol</artifactId>
    <version>1.1.6</version>
</dependency>

The following Java example connects to a running Canal server, subscribes to binlog events, filters for row‑level INSERT or UPDATE operations, and prints the changed column values:

package com.sample.canal;

import com.alibaba.otter.canal.client.CanalConnector;
import com.alibaba.otter.canal.client.CanalConnectors;
import com.alibaba.otter.canal.protocol.CanalEntry;
import com.alibaba.otter.canal.protocol.Message;
import java.net.InetSocketAddress;
import java.util.List;

public class CanalExample {
    public static void main(String[] args) throws Exception {
        CanalConnector canalConnector = CanalConnectors.newSingleConnector(
                new InetSocketAddress("Canal_Service_Address", 11111), "example", "", "");
        canalConnector.connect();
        while (true) {
            canalConnector.subscribe();
            Message message = canalConnector.getWithoutAck(100);
            for (CanalEntry.Entry entry : message.getEntries()) {
                if (entry.getEntryType() == CanalEntry.EntryType.ROWDATA) {
                    CanalEntry.RowChange row = CanalEntry.RowChange.parseFrom(entry.getStoreValue());
                    CanalEntry.EventType eventType = row.getEventType();
                    for (CanalEntry.RowData rowData : row.getRowDatasList()) {
                        if (eventType == CanalEntry.EventType.INSERT || eventType == CanalEntry.EventType.UPDATE) {
                            List
columns = rowData.getAfterColumnsList();
                            printChanges(columns);
                        }
                    }
                }
            }
            canalConnector.ack(message.getId());
        }
    }

    private static void printChanges(List
columns) {
        columns.forEach(column -> {
            String name = column.getName();
            String value = column.getValue();
            System.out.println("Column: " + name + ", New Value: " + value);
        });
    }
}

When a new row is inserted into the monitored table, the program outputs the column names and their updated values, demonstrating real‑time data capture.

In summary, Canal’s deployment is straightforward: by posing as a MySQL slave it can listen to master changes, enabling lightweight, non‑intrusive incremental data synchronization across heterogeneous systems.