How Pika Implements High‑Capacity Redis‑Compatible Replication

Introduction

In the previous article the binlog metadata, format, and related APIs of Pika master‑slave replication were introduced; this article describes the replication‑related threads, the full replication process, and the incremental replication process.

What is Pika

Pika is a large‑capacity Redis‑compatible storage developed by the 360 Web Platform DBA and infrastructure team. It is not meant to replace Redis but to complement Redis in big‑data scenarios by using persistent storage to overcome Redis limitations such as slow recovery, high sync cost, single‑thread fragility, limited data capacity, and high memory cost.

Replication Workflow

Threads

PikaBinlogReceiverThread : Initialized at system start, occupies port+1000, receives Redis commands from the master as a slave.

PikaHeartbeatThread : Initialized at system start, occupies port+2000, receives ping/spci from slaves for liveness detection.

PikaTrysyncThread : No role at start; handles the background tasks of the slaveof host port command, periodically checks connection needs, performs DB replacement, and starts or stops the rsync service on port+3000.

PikaSlavepingThread : After becoming a slave, periodically sends ping/spci to the master; if a timeout exceeds 30 seconds, it closes the master’s background tasks, which are then handled by PikaBinlogReceiverThread.

BinlogBGWorker : Initialized at system start; each contains a binlogbg_thread that executes scheduled Redis commands dispatched by PikaBinlogReceiverThread.

PikaBinlogSenderThread : When acting as a master, consumes log entries based on filenum and offset received from a slave and sends them to the slave’s receiving port.

Class Diagram

Main API Description

Thread

<code>// Initialization
int Thread::InitHandle();

// Periodic task execution
void Thread::CronHandle();

// Create thread, call RunThread()
int Thread::StartThread();

// Logic entry point
void *Thread::RunThread(void *arg);
</code>

HolyThread

<code>virtual int InitHandle()
    // Initialize epoll, add socket to server_fds

virtual void *ThreadMain()
    // Call CronHandle for timed tasks
    // If EPOLLIN on server_fds, accept connection and establish slave‑master link
    // If EPOLLIN on slave‑master link, process Redis command via in_conn->GetRequest()
    // If EPOLLOUT on slave‑master link, send reply via in_conn->SendReply();
</code>

PikaBinlogReceiverThread

<code>void PikaBinlogReceiverThread::CronHandle()
    // Execute background tasks; when PikaSlavepingThread decides to exit or master fails, add Kill PikaBinlogSenderThread task.
</code>

PikaMasterConn

<code>int PikaMasterConn::DealMessage()
    // If monitor exists, duplicate command to monitor (Redis MONITOR)
    // Generate global log serial to ensure ordered writes to slave binlog
    // If read‑only, binlog appended by PikaBinlogReceiverThread; otherwise handled by BinlogBGWorker
</code>

BinlogBGWorker

<code>void Schedule()
    // Start background thread on first schedule
    // Execute specific background task via DoBinlogBG()

void BinlogBGWorker::DoBinlogBG(void* arg)
    // Parse parameters, fetch command
    // If slave not read‑only, acquire global record lock
    // Record binlog, execute command, record slow log
</code>

Slaveof Process Diagram

Slaveof Procedure

If the command is slaveof no one , stop the rsync service, delete the master, set repl_state_ = PIKA_REPL_NO_CONNECT , and role = master.

If the command is slaveof ip port filenum offset , delete binlog entries before filenum ; if filenum exists, pad the file content before offset with spaces.

Update role to slave, set repl_state = PIKA_REPL_CONNECT , and acknowledge success.

PikaTrysyncThread detects repl_state == PIKA_REPL_CONNECT and starts rsync on port+3000 to receive the master’s DB.

The slave establishes a connection to the master and sends auth if required.

Send the trysync command to request data synchronization.

If the master replies kInnerReplWait , set repl_state_ = PIKA_REPL_WAIT_DBSYNC .

The slave waits until an info file appears in db_sync_path , then replaces the DB, updates filenum and offset according to the info, and resets repl_state_ = PIKA_REPL_CONNECT .

Repeat steps 3‑6; when the master replies kInnerReplOk , set repl_state_ = PIKA_REPL_CONNECTING , stop rsync, create PikaSlavepingThread ; after a successful ping, set repl_state_ = PIKA_REPL_CONNECTED , establishing replication.

Trysync Process

Construct a unique slave identifier from IP and port, set stage = SLAVE_ITEM_STAGE_ONE .

If the master cannot find the filenum sent by the slave, perform bgsave to generate a DB snapshot and info, transfer it via rsync, and clean temporary files.

Based on the slave’s IP, port, and the bgsave filenum and offset , build a PikaBinlogSenderThread .

Filter potentially corrupted binlog content.

Update role_ = PIKA_ROLE_MASTER ; on the second PikaSlavepingThread liveness check, send spci ; the master uses PikaBinlogSenderThread to set stage = SLAVE_ITEM_STAGE_TWO , finalizing replication.

Conclusion

In my view, master‑slave replication is the largest and most complex module in Pika. It adopts a LevelDB‑style WAL log; although the log is non‑idempotent due to Redis command semantics, it solves the problem of Redis incremental replication buffers filling up.