Meta engineers describe the challenges of cache invalidation and consistency in large‑scale distributed systems, explain why stale caches are problematic, present their Polaris observability service and consistency‑tracking techniques, and detail how they raised TAO’s cache consistency from six‑nines to ten‑nines.
The article discusses how to achieve reasonable read/write consistency between databases and caches by emphasizing cache expiration, eventual consistency, analyzing the Cache‑Aside pattern, evaluating four update‑order options, recommending the "update‑DB‑then‑delete‑cache" approach, and suggesting async deletion via message queues and distributed locks to handle concurrency and cache breakdown.
This article explains how caching can accelerate read/write performance and reduce backend load, analyzes its benefits and costs, and details practical design patterns such as update policies, granularity control, penetration, bottom‑hole, avalanche, and hot‑key optimizations for reliable high‑performance systems.
This article explains the fundamentals of distributed transactions, covering ACID, CAP and BASE theories, and reviews various consistency solutions such as two‑phase commit, three‑phase commit, TCC, local message tables, MQ approaches and the Seata framework, highlighting their advantages and drawbacks.
This article explains the concepts of strong and weak consistency, their roots in visibility and ordering within the Java Memory Model, and how Java 8’s ConcurrentHashMap uses volatile fields, CAS, and unsafe operations to achieve strong consistency for put/get while retaining weak consistency for operations like size().
The article explains the CAP theorem’s three properties—consistency, availability, and partition tolerance—its implications for distributed systems, highlights its limitations, introduces the PACELC extension that adds latency versus consistency trade‑offs when no partition exists, and provides real‑world database examples.
This article explains the design principles, six core mechanisms, asynchronous replication, periodic synchronization, new‑node sync, and local read strategy of Nacos's Distro consistency protocol, illustrating how AP/CP choices and the JRaft protocol ensure high availability in a distributed service registry.
The article explains why high‑traffic systems store data in databases but use fast caches like Redis, outlines the inherent consistency problems between cache and database, evaluates four update‑delete strategies with concurrency scenarios, and recommends practical solutions such as post‑update cache eviction, delayed double deletion, expiration policies, message queues, and binlog listeners to achieve eventual consistency.
This article explains why caching is essential for high‑read workloads, describes the consistency challenges when data resides in both MySQL and Redis, compares four cache‑update strategies, and offers practical recommendations such as using expiration, delayed double‑delete, message queues, and binlog subscription to achieve eventual consistency.
The article revisits the CAP theorem, explains linearizable consistency and strict availability, demonstrates with concrete data‑center examples why the CP/AP split is often misleading, and argues that latency concerns and broader failure modes make the binary classification of distributed systems impractical.
This article examines the problem of data inconsistency when writing to both a database and a cache, outlines four typical dual‑write strategies, analyzes their drawbacks in high‑concurrency scenarios, and recommends the most reliable approach—writing to the database first followed by cache deletion with retry mechanisms.
This article explains why caching is essential for reducing CPU and I/O pressure, outlines key cache attributes such as throughput and hit rate, compares popular local cache libraries, describes distributed cache options, and details the design, consistency, monitoring, and hot‑key handling of a transparent multilevel cache architecture.
This article examines the challenges of maintaining consistency between databases and caches during high‑concurrency writes, compares four common update strategies, highlights their pitfalls, and recommends the most reliable approach with practical retry mechanisms.
The article analyzes various cache update strategies—including delete‑then‑write, write‑then‑delete, and asynchronous binlog subscription—examining their impact on system throughput, concurrency safety, failure scenarios, and fault detection to ensure data consistency between cache and database.
The article explains how distributed systems maintain request consistency by employing idempotent IDs, state‑machine tracking, snapshot recording, and coordinated error‑handling strategies—including retries, queries, and messaging—while also addressing read/write separation limits, reconciliation, and disaster‑recovery measures to prevent duplicate actions and state divergence.
This article explains the core principles and a range of consistency protocols—including CAP, ACID, BASE, 2PC, 3PC, Paxos, Raft, Gossip, NWR, Quorum, and Lease—detailing their roles, mechanisms, trade‑offs, and typical use cases in distributed systems.
This article explains the fundamental cache‑aside pattern, why lazy cache deletion is preferred over immediate updates, analyzes simple and complex cache‑database inconsistency scenarios, and proposes a high‑concurrency queue‑based solution with practical considerations for backend systems.
The article analyzes the problem of data inconsistency when using caches such as Redis or local memory alongside a database, classifies data by real‑time requirements, compares four write‑through/write‑behind strategies, and proposes practical solutions like delayed double‑delete, message‑queue compensation, and binlog‑driven cache updates.
This article explains the concept of consistency in distributed transactions, compares strong and weak consistency, relates it to the CAP theorem, and explores theoretical and practical approaches—including XA, NewSQL, and MVCC—to achieve stronger consistency across databases and micro‑services.
This article examines the pitfalls of writing to multiple data stores simultaneously—such as consistency and atomicity problems—and proposes a generic solution using change data capture and message queues to keep databases, Redis, Elasticsearch, and Hadoop synchronized.
The article outlines four key dimensions for evaluating data warehouse data quality—correctness, completeness, timeliness, and consistency—explains common consistency issues such as differing metric values across models, cross‑dimensional aggregations, and real‑time versus batch calculations, and proposes organizational and review mechanisms to mitigate these problems.
The article explains the CAP theorem’s origin, formal proof, detailed definitions of consistency, availability and partition tolerance, its impact on relational and NoSQL databases, various consistency models, and practical trade‑offs for designing distributed data systems.
This article explains the origin of the CAP theorem, its formal definitions of consistency, availability, and partition tolerance, examines how it applies to relational and NoSQL databases, discusses trade‑offs and practical strategies for balancing these properties in distributed system design.
This article explains how data consistency is achieved through local ACID transactions, global XA/2PC mechanisms, CAP and BASE theories, and practical patterns such as reliable events, TCC, and SAGA, providing a comprehensive overview for architects designing distributed systems.
This article explains the concepts of sequential, causal, and eventual consistency models, illustrating their definitions, guarantees, and practical examples—including process interactions and real‑world scenarios—while comparing their strengths, trade‑offs, and related consistency guarantees such as monotonic reads, writes, and read‑your‑writes.
The article explains consistency concepts in distributed systems—including strong and weak (eventual) consistency, CAP and FLP theorems, and key protocols such as 2PC, 3PC, and Paxos—detailing their mechanisms, advantages, and challenges.
This article provides a thorough overview of distributed systems, explaining their definition, core concepts such as partition and replication, key challenges, essential characteristics, typical components and protocols, a practical request flow example, and a curated list of real‑world implementations to help readers build a solid learning roadmap.
The article explains how distributed database systems appear as a single logical database but operate over a network of nodes, detailing failures, stale reads, consistency models, architecture anomalies, sharding, replication, and various read/write scenarios that illustrate the complexity of achieving strong consistency.
This article explains the fundamentals of Jepsen testing, compares it with TLA+, describes its architecture and workflow, illustrates how to apply Jepsen for linearizability verification of distributed systems such as locks, and offers practical guidance on integrating Jepsen or building custom testing frameworks.
This article explains why database capacity planning is challenging and presents a systematic approach—including workload analysis, performance modeling, consistency considerations, and node scaling decisions—using the open‑source NoSQL database ScyllaDB to guide accurate capacity estimation.
This article explains why sizing a database cluster is challenging, outlines a step‑by‑step methodology for estimating workload, configuration and performance, discusses the impact of consistency levels, secondary indexes, materialized views and maintenance, and shows how ScyllaDB can be used to model and simplify capacity planning.
This article introduces core concepts of distributed systems, including node and replica models, various consistency levels, data distribution strategies, lease and quorum mechanisms, replica control protocols such as primary‑secondary, two‑phase commit, MVCC, Paxos, and the CAP theorem, providing a comprehensive overview for architects.
This article provides a comprehensive overview of distributed system fundamentals, covering system models, node concepts, failure types, replication strategies, consistency levels, data distribution methods, replica control protocols such as primary‑secondary, two‑phase commit, lease‑based caching, quorum, MVCC, Paxos, and the CAP theorem.
This article details Ctrip Finance's design and implementation of a unified high‑availability Redis‑based distributed cache, covering eventual and strong consistency scenarios, multi‑region deployment, lock mechanisms, message‑driven updates, failure handling, and performance results.
This article provides a detailed summary of database concepts including data models (relational, document, graph), storage engine architectures (page‑oriented B‑tree, log‑structured LSM), transaction mechanisms, isolation levels, distributed transaction protocols, partitioning strategies, indexing, consistency models, and consensus algorithms such as Lamport timestamps and Raft.
The article explains four classic cache‑update design patterns—Cache Aside, Read Through, Write Through, and Write Behind—detailing their workflows, consistency trade‑offs, and practical optimizations for high‑concurrency backend services that combine Redis with MySQL.
This article explains how to maintain data consistency between Redis cache and MySQL database by describing strong, weak, and eventual consistency, three classic cache patterns (Cache‑Aside, Read‑Through/Write‑Through, Write‑Behind), and practical solutions such as delayed double deletion, retry mechanisms, and binlog‑based asynchronous eviction.
The talk explains that achieving consistency in development processes, tool platforms, and personnel capabilities is essential for scaling software organizations without losing productivity, describing entropy, random complexity, the EPC model, and CLCT metric as practical ways to improve R&D efficiency.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements such as POSIX compliance, persistence, scalability, and security, and comparing centralized (e.g., GFS) and decentralized (e.g., Ceph) architectures along with strategies for high availability, performance optimization, and data consistency.
This article provides a comprehensive overview of distributed system fundamentals, covering concepts such as nodes, replicas, consistency models, data distribution strategies, lease and quorum mechanisms, two‑phase commit, MVCC, Paxos, and the CAP theorem, while discussing their practical engineering trade‑offs and failure handling.
This article provides a comprehensive overview of distributed system fundamentals, covering system models, replicas, performance and availability metrics, data distribution strategies, replica protocols such as lease, quorum, two‑phase commit, MVCC, Paxos, and the CAP theorem, with practical engineering considerations.
The article explains the differences between in‑process caching and external cache services such as Redis, discusses when to replace process caches with a cache service, and outlines strategies for maintaining consistency across multiple instances in modern micro‑service architectures.
This article explains the Raft consensus algorithm’s log format, the step‑by‑step log replication process, how leaders ensure consistency through forced overwrites, and details the AppendEntries RPC parameters, illustrated with diagrams and examples.
This article presents Ctrip Finance's design of a unified high‑availability Redis cache service, covering both eventual‑consistency and strong‑consistency scenarios, the overall architecture, data‑accuracy, completeness and availability mechanisms, lock handling, fault‑tolerant updates, and operational recovery strategies.
This article explains the core principles of distributed consistency, covering CAP, ACID, BASE, and detailed walkthroughs of 2PC, 3PC, Paxos, Raft, Gossip, NWR, Quorum, and Lease mechanisms, highlighting their trade‑offs, failure scenarios, and practical usage in modern systems.
This article examines the challenges of caching low‑frequency static data in microservice systems and proposes a comprehensive solution that combines business services, Redis, persistent queues, and consistency‑checking programs to achieve high‑performance, near‑real‑time data access while mitigating cache‑related risks.
This article provides a comprehensive overview of distributed file systems, covering their historical evolution, essential requirements, architectural models with and without central nodes, persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional considerations such as space allocation, file deletion, small‑file handling, and deduplication.
The article explains the origins of the CAP theorem, defines its three core properties, illustrates why a distributed system can satisfy at most two of them, discusses practical workarounds such as eventual consistency, and provides references for further study.
This article explores the core concepts, design requirements, architectural models, scalability, high availability, performance optimization, and security considerations of distributed file systems, comparing centralized and decentralized approaches while highlighting practical solutions for persistence, consistency, and fault tolerance.
This article explains cache‑database consistency challenges, outlines three classic caching patterns, compares delete‑versus‑update strategies, discusses operation ordering, and presents three practical solutions—including delayed double delete, retry mechanisms, and binlog‑based asynchronous eviction—to help maintain data integrity in distributed systems.
This article explains the fundamentals of distributed system consistency, covering weak and strong consistency, the CAP theorem, ACID vs BASE, and detailed overviews of protocols such as 2PC, 3PC, Paxos, Raft, Gossip, NWR, Quorum, and Lease mechanisms.
This article explains the concepts of strong, weak, and eventual consistency, introduces three classic cache patterns (Cache‑Aside, Read‑Through/Write‑Through, Write‑Behind), and discusses practical strategies such as delayed double‑delete, retry mechanisms, and binlog‑based asynchronous eviction to keep Redis and MySQL data in sync.
The article explains Youzan’s Transparent Multilevel Cache (TMC), detailing its architecture, hotspot detection, local caching, consistency mechanisms, and real‑world performance gains during flash‑sale events, showing how it reduces cache pressure and improves latency for Java‑based services.
The talk traces the evolution of distributed databases—from early shared‑memory and shared‑nothing designs through New‑SQL breakthroughs—to Tencent’s HTAP‑oriented TDSQL, highlighting its strong‑consistency architecture, anomaly taxonomy, and performance advantages over systems like Google Spanner while addressing modern scalability and hardware challenges.
The article explains the importance of data quality in the big data era, defines key quality dimensions such as completeness, uniqueness, validity, consistency, accuracy, timeliness, and credibility, and details how each dimension can be measured and enforced through specific constraints and validation rules.
This article provides a comprehensive overview of database system design, covering the historical evolution, classification of relational and NoSQL databases, common architectures, consistency models, indexing techniques, storage formats, compression, and practical considerations for selecting and optimizing database solutions.
This article provides a comprehensive overview of distributed system fundamentals, covering node models, replica concepts, various consistency levels, data distribution strategies, lease-based caching, quorum mechanisms, two‑phase commit, MVCC, Paxos consensus, and the CAP theorem, illustrating each with practical examples and diagrams.
This comprehensive guide covers the core concepts of distributed systems—including CAP theory, consistency models, transaction protocols like 2PC/3PC and TCC, Redis features, persistence options, expiration strategies, memory eviction policies, distributed locking with Redis and Zookeeper, as well as an overview of Kafka's architecture and reliability mechanisms.
This article explains the importance of distributed transactions in micro‑service architectures, reviews ACID properties, compares various solutions such as 2PC/3PC, XA, local message tables, transactional messages, TCC, Saga, and introduces the Seata framework with its AT, TCC, and Saga modes, while also discussing consistency versus consensus with Paxos.
This article examines cache consistency challenges in Redis-backed systems and compares several update strategies—including TTL, delayed double‑delete, cache‑aside, and message‑queue approaches—detailing their workflows, code examples, advantages, and drawbacks to guide backend developers toward reliable cache invalidation.
This article explores the fundamentals of distributed transactions in microservice architectures, detailing ACID properties, classic protocols like 2PC/3PC and XA, modern patterns such as TCC, Saga, local message tables, transaction messages, and the Seata framework, while comparing their trade‑offs with Paxos consensus.
This article outlines core database architecture principles, compares four common deployment schemes (master‑slave, dual‑master, master‑slave with read/write separation, and hybrid), and presents practical consistency solutions for both primary‑replica and DB‑cache mismatches, followed by personal evolution insights.
This article examines core MySQL architecture principles—high availability, performance, consistency, and scalability—then compares four common deployment patterns, analyzes their trade‑offs, and presents practical consistency solutions for both master‑slave replication and cache synchronization, ending with personal insights on evolution and sharding.
This article explains why distributed theory and algorithms are crucial for architects, outlines the four foundational theories and eight key protocols, discusses their four evaluation dimensions, and provides a step‑by‑step learning roadmap illustrated with stories and practical examples.
This article explains how the HBase hbck tool verifies region consistency and table integrity, lists the most frequently used hbck commands, and describes both low‑risk and high‑risk repair procedures, including handling of RIT states and log‑based troubleshooting.
This article uses a mythological alchemy dialogue to illustrate distributed consistency concepts, mapping alchemical furnaces to server nodes and explaining the Quorum NWR protocol, its parameters N, W, R, and how they affect strong and eventual consistency in database systems.
This article explains why ZooKeeper was created, its role in simplifying distributed application development, key characteristics such as high performance, high availability, and consistency, and outlines the fundamental problems it solves in multi‑process coordination across networked systems.
From CAP and BASE trade‑offs to the rigorous Paxos consensus and the more approachable Raft protocol, this article explains how modern distributed systems achieve consistency despite partitions, failures, and latency, detailing roles, phases, and safety guarantees that underpin reliable micro‑service architectures.
This article outlines the background, composition issues, and a three‑principle design system—consistency, interaction reasonableness, and flexibility—used to create a unified UI for 58’s multi‑business‑line messaging center micro‑chat, detailing component specifications, navigation rules, typography, colors, and layout guidelines.
This article provides a comprehensive overview of the Raft consensus algorithm, detailing node roles (follower, candidate, leader), the leader election process, mechanisms ensuring cluster consistency, handling of node failures, and strategies for resolving split‑brain scenarios in distributed systems.
This article explains the role of timestamps in distributed transaction processing, comparing implementations such as MVCC‑based TSO, Google’s TrueTime, Lamport logical clocks, and Hybrid Logical Clocks, and discusses their impact on linearizability, consistency guarantees, and practical designs like TiDB Async Commit.
This article provides a comprehensive overview of distributed system fundamentals, covering core concepts such as nodes, replicas, consistency models, data distribution strategies, lease and quorum mechanisms, two‑phase commit, MVCC, Paxos, and the CAP theorem, along with practical considerations for designing robust, scalable services.
This comprehensive guide explores the core concepts of distributed systems, covering node models, replica strategies, consistency levels, data distribution techniques, lease-based caching, quorum mechanisms, logging, two‑phase commit, MVCC, Paxos consensus, and the CAP theorem, providing practical insights for building robust scalable architectures.
This article provides a comprehensive overview of distributed system fundamentals, covering system models, replica concepts, consistency levels, performance metrics, data distribution strategies, basic replica protocols, lease mechanisms, quorum, logging, MVCC, Paxos, and the CAP theorem.
This article examines the challenges of cross‑region distributed consistency, reviews industry solutions such as direct deployment, learner roles, and partitioned services, and introduces Alibaba’s log‑mirroring decoupled architecture, evaluating trade‑offs in latency, scalability, availability, and consistency for global systems.
This article outlines core database architecture principles—high availability, performance, consistency, and scalability—and evaluates four common schemes (primary‑backup, dual‑master, master‑slave with read/write separation, and hybrid), followed by detailed consistency‑resolution strategies and personal insights.
This article explains ZooKeeper's purpose, core features, and design goals, showing how it simplifies distributed application development by providing high‑performance, highly available coordination services such as naming, locks, leader election, and configuration management while hiding low‑level complexities.
This article explains how to design a flash‑sale system that handles massive concurrent requests by focusing on high performance through dynamic‑static separation, hotspot optimization, and code‑level tuning, while ensuring strong consistency for inventory and maintaining high availability via traffic shaping, fault‑tolerance, and operational best practices.
ZooKeeper is an open‑source distributed coordination service that provides primitives for synchronization, configuration management, and naming, featuring a hierarchical data model of znodes, session handling, one‑time watches, strong consistency guarantees, leader election via Zab protocol, and detailed roles of leader, follower, and observer.
This article explains the challenges of distributed systems such as node failures and network anomalies, then introduces the CAP theorem, BASE theory, two‑phase and three‑phase commit protocols, and details consensus algorithms including Paxos, Raft, ZAB, and Amazon Dynamo's NWR model, highlighting their trade‑offs and practical usage.
The article compares Eureka and Zookeeper as service registry solutions, explaining how Eureka follows an AP model with high availability and eventual consistency, while Zookeeper adopts a CP model prioritizing strong consistency, and discusses their suitable scenarios, limitations, and design considerations for distributed systems.
This article explains the concept of consistency in distributed systems, distinguishes strong and weak consistency, outlines common use cases and challenges, introduces the CAP theorem, and details several consensus protocols including 2‑phase commit, 3‑phase commit, and the Paxos algorithm.
This article explains a real‑world case where a newly added mapping rule didn't appear immediately due to master‑slave replication lag, walks through the evolution of database architectures from single‑node to read‑write separation, and presents several practical strategies to mitigate consistency issues caused by replication delay.
The article thoroughly explains Kafka’s message‑queue reliability design and implementation, covering use‑case scenarios, core concepts, storage format, producer acknowledgment settings, broker replication mechanisms (ISR, HW, LEO), consumer delivery semantics, the epoch solution for synchronization, and practical configuration guidelines for various consistency and availability requirements.
This article examines the evolution, core requirements, architectural models (centralized and decentralized), persistence strategies, scalability, high availability, performance optimization, security mechanisms, and additional design trade‑offs of distributed file systems, providing a comprehensive overview for architects and engineers.
This article explores common data‑synchronization challenges such as double‑write consistency and atomicity issues across databases, Redis, Elasticsearch and Hadoop, and presents a generic solution using ordered message queues and change‑data‑capture middleware to ensure reliable, consistent updates.
This article explains why data consistency is essential in distributed systems, defines consistency, compares strong and eventual consistency, and details the design and operation of major algorithms such as Paxos, Multi‑Paxos, Raft, ZAB, and Gossip with illustrative examples and diagrams.
This article examines the motivations for service discovery in microservice architectures, compares client‑side and server‑side discovery patterns, discusses consistency versus availability trade‑offs (CAP, CP vs AP), and reviews various health‑check mechanisms, graceful up/down strategies, metadata considerations, and high‑availability designs.
This article uses a relatable memory‑service scenario to illustrate the CAP theorem, explaining how consistency, availability, and partition tolerance cannot all be achieved simultaneously in distributed systems and exploring practical trade‑offs through successive design attempts.
Facing scalability limits, disaster risks, and latency in single‑region setups, OPPO built the JinS data‑sync framework, detailing its multi‑region challenges, design principles, modular architecture, consistency models, crash‑safe 2PC mechanisms, file‑queue optimizations, relay‑log handling, and performance gains over open‑source and commercial alternatives.
This article explains the CAP theorem, its trade‑offs, the BASE model, various consistency levels, and how parameters N, W, and R are used to achieve strong or eventual consistency in distributed database systems.
The article explains how distributed cache‑aside patterns work, why deleting stale cache entries is often preferable to updating them, analyzes basic and complex cache‑database inconsistency scenarios, and proposes a JVM‑queue‑driven, single‑threaded update mechanism with practical considerations for high‑concurrency environments.
This article explains how concurrency issues that appear in single‑threaded programs become amplified in distributed systems, covering consistency models, network reliability, clock synchronization, fault detection, backpressure, and cascading failures, and offers practical design and testing strategies to build resilient architectures.
The article explains the classic Cache Aside pattern, why deleting rather than updating cache is preferred, analyzes basic and complex cache inconsistency scenarios, and proposes a queue‑based lazy update solution with practical considerations for read‑write blocking, request routing, and hotspot handling in high‑traffic systems.
The article explains the evolution of database architecture toward distributed SQL, outlining its seven core features such as scalability, consistency, elasticity, geo‑replication, SQL support, data locality, and multi‑cloud capability, while also describing essential database functions and practical considerations for cloud‑native deployments.
Distributed SQL databases combine the reliability of traditional relational systems with cloud-native scalability, offering seven core traits—scalability, consistency, elasticity, geo-replication, SQL support, data locality, and multi‑cloud operation—while still meeting essential database functions such as manageability, optimization, security, and integration.
This article explains the cache‑aside pattern, why deleting cache entries is often preferable to updating them, outlines basic and complex cache‑database inconsistency scenarios, and presents a queue‑driven approach with practical considerations for maintaining data consistency in high‑concurrency backend systems.
This comprehensive article explains the core concepts of distributed systems—including node modeling, failure types, replica strategies, consistency levels, performance metrics, data distribution techniques, lease mechanisms, quorum, logging, two‑phase commit, MVCC, Paxos, and the CAP theorem—providing a solid foundation for designing robust, scalable architectures.
This article provides a comprehensive overview of ZooKeeper, covering its role as a distributed coordination service, consistency guarantees, ZAB protocol, Znode types, watcher mechanism, ACL permissions, chroot feature, session management, server roles, data synchronization, deployment modes, and typical use cases in distributed systems.
This article explains the classic Cache‑Aside pattern, analyzes common cache‑database consistency problems in high‑concurrency scenarios, and presents a lazy‑update queue solution that deletes stale cache entries, routes updates through internal JVM queues, and mitigates read‑blocking and hotspot issues.
