This article describes Alipay’s client‑side high‑availability strategy that combines offline risk mining, function‑interface “minesweeping”, RPC/config/jsapi checks, and automated fuzz testing on Android and iOS to detect and eliminate crash‑inducing bugs before release.
This article provides a comprehensive overview of Linux Virtual Server (LVS), covering its basic concepts, three‑tier architecture, load‑balancing techniques, scheduling algorithms, high availability, reliability, and suitable deployment environments for building high‑performance, scalable server clusters.
This article explains the design of a distributed timer service using a time‑wheel data structure stored in Redis, covering application scenarios, required features, architecture components such as access layer, scheduler, worker, and management center, and detailing reliability and performance techniques.
This article explains why Redis needs high availability, defines common failure scenarios, compares several HA architectures—including single‑instance, master‑slave with one or multiple Sentinel processes, and VIP‑based solutions—and provides step‑by‑step guidance for deploying a robust Redis Sentinel cluster.
This article explores how to transform product requirements into robust technical architectures by outlining strategic principles—appropriateness, simplicity, evolution—and tactical guidelines such as high concurrency, high availability, and business design, while addressing uncertainty, component complexity, and practical implementation with Java‑centric examples.
This article explains the fundamentals, architecture, deployment patterns, recovery procedures, performance tuning, and common troubleshooting of MySQL Group Replication (MGR) as a high‑availability solution for private deployments, illustrated with real‑world examples and practical SQL commands.
This article explains how to design robust technical architectures by applying strategic principles of suitability, simplicity, and evolution, and tactical principles covering high concurrency, high availability, and business design, while illustrating logical and physical architecture diagrams and offering practical implementation advice.
Cloud‑native storage is a set of storage technologies designed for cloud‑native environments, offering high availability, scalability, performance, consistency, durability, and dynamic deployment options across public, private, and on‑premises solutions, and addressing the unique challenges of stateful applications running on Kubernetes and similar platforms.
This article explains how to build a highly available Redis service using Redis Sentinel, covering failure scenarios, design principles, four architectural options, the final three‑Sentinel solution with virtual IP, and practical operational tips for ensuring continuous service.
This article details Meitu's adoption of the Presto ad‑hoc ROLAP engine, comparing it with Hive on Spark and Impala, describing two coordinator high‑availability solutions, and explaining the cross‑cluster scheduling architecture that leverages idle Presto resources to improve overall big‑data processing efficiency.
JD's OLAP platform runs on ClickHouse and Doris across 3,000 servers, handling billions of daily queries and petabytes of data, and this article details the selection criteria, cluster deployment models, high‑availability architecture, operational challenges, and future roadmap.
This article presents a comprehensive, step‑by‑step guide on improving system reliability through early fault detection, scope reduction, frequency reduction, and rapid incident handling, using real‑world practices from Baidu's commercial hosting platform.
This article explains how to build a highly available, scalable system by decoupling components, adding redundancy, choosing appropriate scaling strategies, employing load balancers, caching, CDN, database replication, sharding, and stateless design to support over 100 million users.
The article examines Kafka’s high‑availability mechanisms, detailing its multi‑replica design, ISR synchronization, leader election, and the critical role of the __consumer_offset topic, and explains why a single broker outage can render the entire cluster unusable unless replication factors are properly configured.
This guide explains how to combine the open‑source tools Keepalived and HAProxy to create a highly available software load‑balancing solution, covering the underlying concepts, installation steps, configuration files, health‑check scripts, session persistence, SSL offloading, and traffic routing techniques.
The Xianyu recommendation service was re‑architected into an active‑active, multi‑region high‑availability system—using a unified access‑layer router, centralizing long‑tail dependencies, keeping data unsharded, refactoring caches and MySQL replication, and adhering to traffic‑closed‑loop and availability‑first principles—to overcome latency, improve scalability, and ensure low‑cost disaster recovery across two regions and three data centers.
This article provides a comprehensive overview of large‑scale distributed website architecture, covering its characteristics, design goals, architectural patterns, performance, high‑availability, scalability, extensibility, security, agility, evolution stages, and practical implementation techniques such as caching, load balancing, database sharding, service‑orientation and message queues.
This article examines why failure is inevitable in software systems, proposes a mindset of failure‑oriented design, outlines organizational roles and processes to mitigate incidents, and presents concrete technical techniques such as distributed locking and traffic shaping to build resilient, high‑availability services.
This guide explains how to configure HAProxy for high‑performance TCP/HTTP load balancing and combine it with Keepalived to achieve high‑availability using VRRP, covering installation, core features, health checks, session persistence, SSL offloading, routing rules, and practical configuration examples.
This article explains Kafka's high‑availability architecture, covering multi‑replica redundancy, ISR synchronization, producer ACK settings, and the critical role of the __consumer_offset topic, and shows how to configure replication factors to prevent a single‑node outage from stopping consumption.
This guide explains what Redis Sentinel is, its architecture, how to configure it, and demonstrates both slave and master failure recovery as well as configuring multiple Sentinel instances for high availability.
This article details the design and implementation of a high‑performance, highly available membership system for the merged Tongcheng‑eLong platform, covering Elasticsearch dual‑center clusters, traffic‑isolated three‑cluster architecture, deep ES optimizations, Redis caching and dual‑center clusters, MySQL dual‑center partitioning, migration strategies, and future fine‑grained flow‑control and degradation measures.
This article provides a comprehensive overview of Redis, covering its definition, basic and special data structures, performance optimizations, expiration and eviction policies, common application scenarios, persistence mechanisms, high‑availability architectures, distributed lock implementations, transaction handling, and related algorithms such as Redlock and Bloom filters.
This article explains how to design robust Java backend APIs, covering interface definition, request/response formats, error handling, token generation, digital signing, interceptor chains, rate limiting, HTTPS migration, and strategies for high concurrency and high availability such as load balancing, clustering and caching.
This guide explains what Redis Sentinel is, its architecture, how to configure it in a master‑replica environment, and demonstrates handling both slave and master failures with detailed log examples, plus tips for running multiple Sentinel instances for high availability.
Redis, an open‑source in‑memory data store, serves as a database, cache, and message broker, offering rich data types, persistence, replication, Sentinel, and clustering; this article walks through its evolution from simple caching to high‑availability, distributed architectures, and advanced client features.
This article provides a comprehensive list of typical Redis interview questions covering its features, performance, data structures, supported data types, common use cases, eviction policies, persistence methods, clustering, high‑availability mechanisms, transaction handling, and comparisons with local caches like Guava and Caffeine.
This article presents a detailed interview-style walkthrough of Redis fundamentals, covering its thread model, persistence mechanisms (AOF, RDB, hybrid), command behaviors, high‑availability options, and the hash‑slot based clustering algorithm, illustrated with code snippets and diagrams.
This article details NetBank's evolution of database deployment—from early distributed setups to a unitized, cloud‑native architecture—covering disaster‑recovery upgrades, distributed database design, multi‑tenant strategies, containerized migration, and the performance and operational impacts of moving to a three‑site five‑center model.
MQProxy is a Java‑based distributed message‑queue proxy built on Apache Kafka that abstracts underlying queue selection, protocols, and health monitoring, offering developers a simple SDK with produce/consume/commit APIs, advanced features like delayed and dead‑letter queues, and a scalable architecture for high availability.
This article explains the concept of service degradation, defines SLA levels including the six‑nine metric, and details practical strategies such as fallback data, rate‑limiting, timeout handling, read/write degradation, retry mechanisms, and front‑end techniques to maintain high availability during traffic spikes.
This article details Qunar's design and implementation of a data synchronization platform that aggregates MySQL data into Elasticsearch, supporting parallel ES5.x/7.x clusters, hot‑swap upgrades, flexible scaling, and end‑to‑end consistency with high availability mechanisms.
This article explains the concept of service degradation, its relationship with rate limiting and SLA, and presents various practical mitigation techniques such as fallback data, rate‑limit throttling, timeout handling, fault isolation, retry mechanisms, feature switches, read/write degradation, and front‑end strategies to maintain high availability during traffic spikes or component failures.
This article provides a comprehensive technical overview of large‑scale distributed website architecture, covering its characteristics, design goals, architectural patterns, performance, high availability, scalability, extensibility, security, agility, and a detailed example evolution from a single‑server setup to a multi‑layer, service‑oriented system.
An in‑depth look at Kafka’s high‑availability mechanisms reveals how multi‑replica design, ISR leader election, and the request.required.acks setting interact, why a single broker failure—especially of the __consumer_offset topic—can halt consumption, and how to configure replication factors to prevent such outages.
This article provides a comprehensive overview of Apache RocketMQ, detailing its core components—Namesrv, Broker, Producer, and Consumer—explaining routing registration, message storage, queue allocation strategies, and key concepts such as topics, tags, and consumer types, while comparing it with Kafka.
This article explains what Redis Sentinel is, its architecture, how to configure sentinel.conf, monitor master and slaves, handle slave and master failures, and set up multiple sentinel instances for robust high‑availability in a Redis master‑slave environment.
The article details JD.com’s large‑scale OLAP strategy using ClickHouse as the primary engine and Doris as a secondary engine, covering application scenarios, component selection criteria, cluster deployment models, high‑availability architecture, fault‑handling procedures, performance tuning, and future cloud‑native plans.
The article explains why large‑scale services must conduct production full‑link load testing, describes its evolution from ad‑hoc trials to standardized monthly practices, and details the technical and procedural steps—including traffic modeling, JMeter usage, middleware tagging, and responsibility mapping—that ensure reliable capacity planning and risk mitigation.
This case study details how Car Home leveraged TiDB 5.1.1 with a three‑data‑center, five‑replica HTAP architecture to support the high‑traffic 818 Global Auto Festival, covering background, business requirements, database selection, system design, performance challenges, solutions, and post‑event insights.
Tinyid is a Java‑based distributed ID generator that uses a database segment algorithm, supports multiple master databases, offers both HTTP and Java client interfaces, and provides high throughput and availability for billions of IDs daily.
This article explains the role of distributed architecture for high‑performance internet systems, introduces GreatSQL as a native distributed relational database, compares it with MySQL, and provides step‑by‑step Docker‑Compose instructions to set up, start, and verify a three‑node MGR cluster, plus integration with the PIG microservice platform.
Redis, the high‑performance open‑source key‑value store, uses a master‑slave replication mechanism that ensures data redundancy, read/write separation, fault recovery, and high‑availability; this article explains its handshake process, synchronization phases, replication states, and key source‑code functions in detail.
This guide explains how to achieve high availability for Nginx by installing and configuring Keepalived with VRRP, detailing preparation steps, configuration file examples, health‑check scripts, and failover testing on two Linux servers.
This article explains MongoDB's evolution from Master‑Slave to Replica Set and Sharding architectures, detailing how each model provides high availability, data reliability, and scalability, and offers practical configuration tips to ensure strong consistency and minimal downtime in production deployments.
This article explains Redis Sentinel's role in providing high availability by detailing how Sentinels discover each other, detect master failures, elect a new master, and communicate state using the gossip protocol, offering a clear conceptual overview for database practitioners.
This article presents a comprehensive design for a high‑availability message center that targets 10,000 messages per second inbound throughput and 1,000 messages per second outbound delivery, detailing technical goals, functional requirements, technology selection, architectural diagrams, and implementation guidelines using RocketMQ, Elasticsearch, Spring Cloud Gateway, MySQL, Docker, and Kubernetes.
This article reviews Redis's latest features and compares various deployment architectures—including single‑replica, dual‑replica, cluster, and read‑write‑separation modes—detailing their reliability, performance characteristics, suitable use cases, and provides a Java Jedis example for configuring a direct‑connect cluster.
This article explains the DNS hijacking threat, shares a real incident affecting NetEase Cloud IM, and details a comprehensive high‑availability architecture—including HttpDNS, laddered HTTP requests, caching strategies, and SNI handling—that protects the SDK from DNS attacks and ensures reliable service.
This guide introduces HAProxy, an open‑source L4/L7 load balancer, and walks through its core features, performance and stability characteristics, step‑by‑step installation on CentOS 7, configuration of both L7 and L4 balancing, monitoring, and setting up high‑availability with Keepalived.
This guide explains how to achieve high availability for Nginx by deploying a dual‑machine keepalived setup, covering the concepts of HA, VRRP, configuration of keepalived on master and backup nodes, a health‑check script, and step‑by‑step commands to test automatic failover.
This talk shares practical high‑availability strategies learned from Alibaba’s Taobao platform and Alibaba Cloud, covering traditional IDC stability, cache and disaster‑recovery designs, cloud‑native fault‑tolerance, performance‑capacity trade‑offs, traffic shaping, multi‑region replication, and lessons from real‑world incidents like GitLab failures.
This document outlines a comprehensive SRE‑driven operational framework for ensuring stable, high‑availability online education services during peak summer and winter periods, detailing pre‑, during‑, and post‑maintenance phases, architectural principles, load testing, monitoring, capacity management, safety hardening, chaos engineering, incident response, and post‑mortem practices.
Alibaba’s eight‑year Double 11 journey illustrates how the company tackled exponential business growth by inventing high‑availability middleware, precise capacity planning, unit‑based deployment, online stress testing, hybrid‑cloud elasticity, and intelligent runtime control to balance throughput, cost, and user experience during the midnight peak.
This article explains the concepts of disaster tolerance, fault tolerance, and disaster recovery, compares them with backup strategies, outlines key metrics such as RTO and RPO, and presents common architectures and planning considerations for building resilient enterprise systems.
The article details Meituan’s five‑year evolution of its instant logistics platform, describing the distributed backend architecture, AI‑driven optimization, scalability and high‑availability practices, as well as future challenges in microservice complexity and operational automation.
This article describes how Ctrip's senior backend engineer designed and implemented horizontal database sharding, a service‑level proxy, dual‑read/write mechanisms, and a staged migration process to overcome order‑database bottlenecks, improve scalability, and ensure high availability for the rapidly growing international train‑ticket business.
The article describes how a bank built a Kubernetes‑based, containerized MySQL service platform (CDD) to improve database high availability, resource utilization, automated operations, and agile delivery by addressing network, storage, scheduling, and management challenges through custom networking, hybrid storage, scheduler extensions, and multi‑AZ deployment.
This guide explains how to achieve high availability for Nginx using Keepalived by setting up a double‑machine hot‑standby configuration, installing necessary packages, configuring VRRP‑based failover, writing health‑check scripts, and testing automatic VIP takeover when the primary server fails.
The article explains how enterprises are moving from traditional primary‑backup and two‑region three‑center data‑center models toward distributed active/active data‑center architectures to achieve continuous 24/7 operations, higher resource utilization, and fault‑transparent services, while outlining the technical and organizational challenges involved.
This guide explains how to build a highly available, scalable architecture for supporting hundreds of millions of users, covering decoupling, redundancy, vertical and horizontal scaling, load balancing, database replication, sharding, caching, CDN, GeoDNS, and best practices for progressive system expansion.
This article explains the need for a messaging system, explores Kafka's core concepts, cluster architecture, performance optimizations like sequential disk writes and zero‑copy, and provides detailed guidance on sizing hardware, configuring producers and consumers, and managing a production Kafka deployment.
The article explains the concepts of cache penetration, cache breakdown, and cache avalanche in Redis‑based systems, analyzes the performance problems they cause under high concurrency, and presents practical mitigation techniques such as Bloom filters, caching empty objects, distributed locks, high‑availability clusters, rate limiting, and data pre‑warming.
This article explains the different Cloud MySQL instance architectures, details asynchronous, semi‑synchronous, and strong synchronous replication, describes high‑availability failover mechanisms, outlines upgrade procedures, and covers binlog usage, rollback methods, slow‑query optimization, and storage fragmentation.
Tongcheng Yilong’s BaikalDB deployment combines high‑availability multi‑Raft HA, HTAP support, and share‑nothing scalability to deliver over 72K TPS OLTP and ten‑fold faster OLAP queries while cutting operational costs up to a hundredfold through dual‑center, columnar storage and cloud‑native elasticity.
This guide walks through deploying the NFS‑subdir‑external‑provisioner on Kubernetes, covering migration to the new repository, configuring storage classes with subdirectory support, applying RBAC resources, creating PVCs, enabling high‑availability with leader election, and troubleshooting common mount errors.
This article explains how to transform single‑node Elastic Job Lite deployments into highly available solutions, covering Zookeeper‑based sharding, active‑standby strategies for dual‑data‑center setups, custom sharding implementations, and priority scheduling to ensure tasks run reliably across both primary and backup sites.
This article details the design and architecture of a cloud‑based shopping cart system, covering its functional modules, layered and cluster designs, technical goals such as stability and elasticity, three‑level caching, asynchronous checks, storage heterogeneity, and payment‑processing solutions.
An in‑depth look at Kafka’s multi‑replica architecture, ISR mechanism, and ack settings reveals why a single broker failure—especially of the __consumer_offset topic—can halt consumer reads, and how proper replication factors and configuration can ensure true high‑availability.
The article defines strong versus weak service dependencies, outlines governance through discovery, fault injection, and refactoring, recommends front‑end and back‑end fault‑tolerance measures such as timeouts and circuit breakers, describes isolation and artificial degradation switches, verifies classifications, and notes current middleware gaps and hiring information.
This article describes how Baidu engineered a flexible, multi‑level fault‑tolerant architecture—including dynamic retry scheduling, multi‑recall coordination, ranking layer degradation, and cross‑IDC multi‑master storage—to achieve five‑nine availability for its massive feed recommendation service.
This article explains how Redis Sentinel provides automatic failover, monitoring, and client notification, outlines its configuration, compares three methods for receiving failover messages, and recommends a service‑level indirect reception design for scalable, low‑cost high availability.
This guide explains Tencent Cloud MySQL's three instance types, detailed replication modes (asynchronous, semi‑synchronous, strong synchronous), high‑availability failover mechanisms, upgrade procedures, binlog management, restore options, slow‑query tuning, and space fragmentation, providing practical insights for reliable cloud database operations.
This article explains what Redis Sentinel is, its architecture, how to configure it in a master‑slave environment, and demonstrates handling of slave and master failures as well as setting up multiple Sentinel instances for high availability.
The article explains the concepts of cache penetration, cache breakdown, and cache avalanche in Redis‑based systems, describes the performance risks they pose to persistent databases, and presents practical mitigation techniques such as Bloom filters, empty‑object caching, hot‑key permanence, distributed locks, high‑availability clusters, rate limiting, and data pre‑warming.
This guide demonstrates how to switch a MySQL 5.7.32 deployment between Group Replication (MGR) and semi‑synchronous replication, covering environment checks, node configuration, plugin installation, replication setup, validation, and the limitations encountered when combining the two modes.
This article provides a comprehensive guide to configuring LVS DR mode, including virtual router setup, network addressing, ipvsadm scheduling, firewall marking, persistence mechanisms, and high‑availability health‑check scripts for robust load‑balancing deployments.
This guide walks through essential Kubernetes operations such as isolating and recovering nodes, expanding clusters with new nodes, dynamically scaling Pods, managing Labels, scheduling Pods to specific Nodes, performing rolling updates, and configuring high‑availability for etcd and Master components, all with concrete command‑line examples and YAML snippets.
In this article, a programmer recounts the recent Bilibili outage, analyzes its timeline, proposes technical root‑cause hypotheses such as CDN failure and service‑chain avalanche, shares insights from the platform’s high‑availability architecture, and outlines preventive techniques for building more resilient backend systems.
The article analyzes Bilibili’s recent service outage, explains why high availability matters, introduces key metrics like MTBF and MTTR, and outlines practical strategies such as redundancy, rate limiting, isolation, failover, timeout control, circuit breaking, degradation, and multi‑region deployment to build resilient systems.
This article explains the principles and implementation of content‑based request distribution in Linux IPVS and the kernel‑mode KTCPVS, covering TCP gateway vs. migration, scheduling algorithms, high‑availability mechanisms, and performance benefits such as improved cache hit rates and scalability.
The article compares the architectures of NSQ (YouZan branch), Kafka, and RocketMQ—detailing their coordination mechanisms, storage models, consistency guarantees, and operational trade‑offs—while recommending Kafka for log‑big‑data workloads, RocketMQ for massive topic counts, and NSQ for extensibility and lightweight deployment.
This article explains Alibaba Cloud's Redis read‑write separation architecture, comparing star and chain replication models, their performance and scalability trade‑offs, and how transparent compatibility, high availability, and high performance are achieved through redis‑proxy, HA monitoring, and optimized binlog replication.
This article analyzes Bilibili's recent service disruption, explains the concept and quantitative metrics of high availability, and outlines practical techniques such as rate limiting, isolation, failover, timeout control, circuit breaking, degradation, and multi‑region active‑active deployments to improve system reliability.
This article explains how to move from risky single‑node deployments of Elastic Job Lite to a robust high‑availability setup across two data centers, covering Zookeeper‑based sharding consistency, priority scheduling, and custom sharding strategies with full Java code examples.
This article explains how to design and deploy a highly available Redis architecture using Sentinel, covering failure scenarios, evaluation of common HA solutions, step‑by‑step configurations from a single‑node setup to a three‑Sentinel deployment, and practical tips such as using virtual IPs for seamless client access.
This article analyzes various high‑availability strategies for stateful backend services—covering cold backup, dual‑machine hot standby, same‑city active‑active, remote active‑active, and multi‑region active‑active architectures—detailing their benefits, limitations, and practical implementation considerations.
This article explains the architecture of a cloud‑based shopping cart, covering its functional roles, layered and cluster designs, distributed technical architecture goals, three‑tier caching, storage heterogeneity, payment solutions, Nginx+LUA aggregation, anti‑bot measures, and multi‑dimensional user feature identification.
This article explains the principles and architecture of scalable, high‑availability network services using Linux Virtual Server (LVS), covering definitions, requirements, load‑balancing mechanisms, cluster components, geographic distribution, BGP routing, and practical deployment considerations for web, media, cache, and mail services.
This article outlines the technical, business, and product goals for a messaging center, presents a prototype and functional requirements, evaluates RocketMQ and Elasticsearch as core technologies, and details the architectural design, underlying frameworks, and DevOps strategies—including Spring Cloud Gateway, Kubernetes, and Docker—to achieve 10,000 msg/s upstream throughput, 1,000 msg/s downstream delivery, and 100 % high availability.
This article describes the architecture and optimization of JD.com’s coupon platform, covering the JimDB distributed in‑memory database for core storage, a massive distributed task system for product coupons, high‑availability strategies for store coupons, and the overall middle‑platform design that ensures scalability, low latency, and data consistency across millions of daily transactions.
The article introduces Kuaishou's Havok data‑service platform—a one‑stop, configuration‑driven solution that lowers development barriers—and details the comprehensive high‑availability architecture, including hierarchical isolation, elastic scaling, link grading, disaster recovery, and rate‑limiting mechanisms that enable zero‑failure support for large‑scale events.
This article explains why RocketMQ was chosen, describes its large‑scale usage, details the design and implementation of a same‑city dual‑center architecture with near‑by production and consumption, outlines failover mechanisms, governance practices, lessons learned, and future plans for the messaging platform.
This article defines high availability, explains why achieving four nines is a common goal, and categorizes HA into compute and storage solutions, detailing common architectures such as active‑passive, master‑slave, symmetric and asymmetric clusters, as well as various storage replication patterns.
This article introduces HAProxy as a free, high‑performance load balancer, explains its core L4/L7 features, walks through installation on CentOS 7, shows detailed configuration for HTTP and TCP modes, covers logging, log rotation, health checks, session persistence, monitoring, and demonstrates high‑availability setup using Keepalived.
The presentation details the TalRTC real‑time communication platform used in online education, covering its product overview, three‑layer architecture, high‑availability and weak‑network strategies, as well as special optimizations for teaching scenarios that improve audio‑video quality and reliability.
This article presents Ctrip Finance's design of a unified high‑availability Redis cache service, covering both eventual‑consistency and strong‑consistency scenarios, the overall architecture, data‑accuracy, completeness and availability mechanisms, lock handling, fault‑tolerant updates, and operational recovery strategies.
This article introduces Kuaishou's data platform serviceification, outlining the background challenges for data engineers, the platform's architecture and key technologies such as configuration‑driven development, multi‑mode APIs, data acceleration, and high‑availability mechanisms, and concludes with a summary of achievements and future directions.
This article explains the background, architecture, and replication models of Redis read‑write separation, compares star and chain replication, and outlines its transparent compatibility, high availability, and performance benefits while noting consistency trade‑offs for read‑heavy workloads.
This article examines the difficulties of running stateful services such as Redis, etcd, and MySQL on Kubernetes and presents practical solutions—including workload selection, CRD/operator extensions, scheduling strategies, high‑availability mechanisms, performance‑optimized networking and storage, and chaos‑engineering validation—to achieve reliable, high‑performance containerized deployments.
