data replication

Xiaokun's Architecture Exploration Notes

Jun 8, 2025 · Databases

Why Data Replication Matters: Architectures, Formats, and Consistency Models

This article explores the principles of data replication, comparing shared memory, shared disk, and non‑shared storage architectures, detailing replication formats, consistency challenges, and various replication strategies such as synchronous, asynchronous, semi‑synchronous, and majority‑based approaches, helping engineers choose the right trade‑offs.

asynchronous replicationconsistencydata replication

0 likes · 12 min read

Why Data Replication Matters: Architectures, Formats, and Consistency Models

Cognitive Technology Team

Apr 13, 2025 · Backend Development

Understanding RocketMQ Master‑Slave Architecture and High‑Availability Mechanisms

This article explains how RocketMQ achieves high availability and data reliability through its master‑slave broker design, covering synchronous and asynchronous replication, flush strategies, transaction messaging, automatic failover with Dledger, and read‑write separation for load balancing in distributed systems.

High AvailabilityMaster‑SlaveMessage Queue

0 likes · 7 min read

Understanding RocketMQ Master‑Slave Architecture and High‑Availability Mechanisms

Tencent Cloud Developer

Dec 4, 2024 · Databases

Building a Distributed Database Storage Engine: From LSM Tree to Data Sharding

This article walks through building a database storage engine from a simple shell script to a full distributed key‑value system, covering in‑memory indexing, SSTable creation, LSM‑Tree architecture with compaction, replication strategies, and sharding techniques for scaling across multiple machines.

B+ TreeData ShardingDistributed Database

0 likes · 38 min read

Building a Distributed Database Storage Engine: From LSM Tree to Data Sharding

Efficient Ops

Oct 23, 2024 · Databases

How NineData Boosts R&D Collaboration 5× with Multi‑Cloud Database Management

The NineData presentation at the 2024 GOPS Global Operations Conference in Shanghai detailed multi‑cloud, multi‑source database architecture trends, showcased their intelligent data management platform, explained data replication principles, DevOps challenges and AI‑enhanced solutions, and highlighted real‑world customer success stories across industries.

AIDevOpsMulti-Cloud

0 likes · 11 min read

How NineData Boosts R&D Collaboration 5× with Multi‑Cloud Database Management

Tencent Cloud Developer

Jul 16, 2024 · Big Data

In‑Depth Exploration of Apache Kafka: Architecture, High Reliability, and High Performance

Apache Kafka achieves high‑throughput, fault‑tolerant messaging by combining a partitioned log architecture with leader‑follower replication, asynchronous producer pipelines, configurable acknowledgments, page‑cache‑based sequential writes, zero‑copy transfers, batching, compression, and a multi‑reactor network model that together ensure scalability, reliability, and performance.

Apache KafkaMessage QueueStreaming

0 likes · 30 min read

Architect

Mar 18, 2024 · Databases

MySQL vs PostgreSQL: Overview, Performance Benchmark, and Use‑Case Guidance

The article introduces MySQL and PostgreSQL, compares their performance through a benchmark on identical hardware, discusses each system’s strengths, weaknesses, and suitable application scenarios, and provides guidance on choosing the appropriate database for different workloads.

Database ComparisonMySQLPerformance Benchmark

0 likes · 8 min read

MySQL vs PostgreSQL: Overview, Performance Benchmark, and Use‑Case Guidance

Didi Tech

Nov 14, 2023 · Databases

Didi's Multi-Active Redis Architecture: Design, Challenges, and Solutions

To achieve disaster-recovery and cross-data-center resilience, Didi progressed from a simple proxy double-write scheme to a sophisticated MQ-free multi-active Redis design that uses a dedicated syncer, shard-based loop prevention, op-id replay protection, conflict detection, and incremental AOF durability, ensuring low latency, no data loss, and consistent availability.

DidiHigh AvailabilityRedis

0 likes · 11 min read

Didi's Multi-Active Redis Architecture: Design, Challenges, and Solutions

FunTester

Jun 19, 2023 · Big Data

Kafka Architecture and Core Concepts: Brokers, Producers, Consumers, Topics, Partitions, Replicas, and Reliability

This article provides a comprehensive overview of Kafka's architecture and fundamental concepts, covering its overall structure, key components such as brokers, producers, consumers, topics, partitions, replicas, leader‑follower synchronization, offset handling, message storage at both logical and physical layers, as well as producer and consumer workflows, partition assignment strategies, rebalancing, log management, zero‑copy I/O, and reliability mechanisms.

KafkaMessage QueuesStreaming

0 likes · 22 min read

Kafka Architecture and Core Concepts: Brokers, Producers, Consumers, Topics, Partitions, Replicas, and Reliability

Architects Research Society

Apr 25, 2023 · Fundamentals

Understanding Eventual Consistency and Anti‑Entropy in Distributed Systems

This article explains the concepts of eventual consistency, hinted handoff queues and anti‑entropy in distributed databases, illustrates how they work with XDB Enterprise examples, and shows how AE restores data integrity after node failures or network partitions.

XDB Enterpriseanti-entropydata replication

0 likes · 10 min read

Understanding Eventual Consistency and Anti‑Entropy in Distributed Systems

Aikesheng Open Source Community

Jan 10, 2023 · Databases

Cassandra Multi‑Data‑Center Fault Tolerance Experiment and Analysis

This article presents a step‑by‑step experiment on a Cassandra cluster spanning two data centers, demonstrating how token ownership, data distribution, and fault‑tolerance behave when nodes fail or are removed, and explains the observed owns percentages and replication effects.

CassandraNoSQLdata replication

0 likes · 15 min read

Cassandra Multi‑Data‑Center Fault Tolerance Experiment and Analysis

Architects' Tech Alliance

Jan 6, 2023 · Operations

Fundamentals of Data Replication, Backup, and Disaster Recovery

This article explains the core concepts of data replication, backup strategies, and disaster recovery, covering RTO/RPO metrics, backup types, copy data management, and the differences between data‑level, application‑level, and business‑level disaster recovery solutions.

Business ContinuityRPORTO

0 likes · 14 min read

Fundamentals of Data Replication, Backup, and Disaster Recovery

Architects' Tech Alliance

Nov 5, 2022 · Databases

Data Replication: Fundamentals, Technologies, and Future Trends

This article explains the concept of data replication, its three-stage process, key principles of compliance, timeliness, and diversity, various replication methods, layered technologies across storage, operating system, and database levels, emerging cloud and big‑data solutions, and heterogeneous use‑case scenarios.

Cloud ComputingStoragebig data

0 likes · 15 min read

Data Replication: Fundamentals, Technologies, and Future Trends

DataFunTalk

Sep 4, 2022 · Big Data

Design and Implementation of Bilibili's Offline Multi‑Datacenter Solution

This article describes Bilibili's offline multi‑datacenter architecture, explaining why a scale‑out approach was chosen over scale‑up, and detailing the unit‑based design, job placement, data replication, routing, versioning, bandwidth throttling, traffic analysis, and the operational results and future directions.

HDFSJob Schedulingbandwidth optimization

0 likes · 24 min read

Design and Implementation of Bilibili's Offline Multi‑Datacenter Solution

Architects' Tech Alliance

Aug 28, 2022 · Databases

Data Replication: Fundamentals, Technologies, and Industry Trends

The article explains data replication concepts, processes, and technologies across storage hardware, operating system, and database layers, outlines synchronous, asynchronous, and hybrid methods, discusses industry applications, trends such as hardware‑software decoupling, cloud replication, and big‑data real‑time copying, and highlights challenges and future directions.

Storagebig datacloud

0 likes · 14 min read

Data Replication: Fundamentals, Technologies, and Industry Trends

Efficient Ops

Jul 19, 2022 · Databases

How CDC Powers Real-Time Analytics Without Overloading Your Database

This article introduces the practice of Change Data Capture (CDC), explaining how capturing only data changes can feed downstream systems and data warehouses in near real‑time, reducing load on the source database, improving reporting latency, and supporting scalable, reliable analytics pipelines.

CDCChange Data CaptureReal-time Analytics

0 likes · 9 min read

How CDC Powers Real-Time Analytics Without Overloading Your Database

Bilibili Tech

Jul 5, 2022 · Big Data

Multi‑Datacenter Architecture for Offline Big Data Processing at Bilibili

To overcome rapid data growth and on‑premise capacity limits, Bilibili adopted a scale‑out, unit‑based multi‑datacenter architecture that isolates failures, intelligently places jobs, replicates data via an enhanced DistCp service, routes reads with an IP‑aware HDFS router, and throttles cross‑site traffic, enabling stable offline big‑data processing of hundreds of petabytes while preserving throughput.

HDFSYARNbandwidth optimization

0 likes · 28 min read

Multi‑Datacenter Architecture for Offline Big Data Processing at Bilibili

Top Architect

May 11, 2022 · Databases

An Introduction to Change Data Capture (CDC) Practices

This article introduces the concept and practice of Change Data Capture (CDC), explaining why CDC is needed for real‑time analytics, how it works by capturing DML changes, modern approaches using transaction logs, and key considerations for building a production‑ready CDC system.

CDCChange Data Capturedata integration

0 likes · 8 min read

An Introduction to Change Data Capture (CDC) Practices

Aikesheng Open Source Community

Apr 19, 2022 · Databases

DTLE 4.22.04.0 Release Notes – New Features and Fixes

The DTLE 4.22.04.0 release introduces UTF‑32 support, Chinese table name handling, enhanced ColumnMapFrom conversion, expanded SqlFilter capabilities, additional logging, and numerous bug fixes including procedure handling, DDL panic prevention, and task list display issues, with links to documentation and related articles.

Bug FixesDTLEFeature Updates

0 likes · 4 min read

DTLE 4.22.04.0 Release Notes – New Features and Fixes

Aikesheng Open Source Community

Apr 12, 2022 · Databases

Using pt-osc for Table DDL Changes with DTLE: Procedure and Support

This article explains how DTLE supports pt-osc table schema changes by describing pt-osc's underlying mechanism, DTLE's DDL capabilities, and step‑by‑step instructions for deploying DTLE, preparing MySQL data, configuring a DTLE job, running pt‑online‑schema‑change, and verifying successful synchronization.

DDLDTLEMySQL

0 likes · 7 min read

Using pt-osc for Table DDL Changes with DTLE: Procedure and Support

Top Architect

Mar 22, 2022 · Big Data

Elasticsearch Cluster Architecture and Data Layer Design

This article explains Elasticsearch's cluster architecture, including nodes, indices, shards, and replicas, compares mixed and tiered deployment models, discusses the data storage layer and replication trade‑offs, and presents two typical distributed data system designs with their advantages and drawbacks.

Cluster ArchitectureShardingbig data

0 likes · 14 min read

Elasticsearch Cluster Architecture and Data Layer Design