1 views collected around this technical thread.
This article explains how Baidu's vertical search platform tackled massive data growth and scaling challenges by redesigning its data management system, introducing elastic scheduling, decoupling ETCD access, implementing auto‑scaling, and advancing shard expansion to improve performance, stability, and cost efficiency.
This article explains how to locate and analyze MySQL slow queries, understand index structures and types, apply best practices for index creation, handle large pagination, manage transaction isolation and MVCC, and implement master‑slave replication and sharding strategies for high‑performance databases.
This article explains a metadata‑driven approach for extending massive tables—using a separate extension table, sharding, and Elasticsearch sync—to add new fields to billion‑row databases without locking the primary table or disrupting online services.
This article explains how to assess MySQL table size from table capacity, disk usage, and instance limits, describes why large tables degrade performance due to B+‑tree depth, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to improve query speed and manage growth.
This article walks through the full process of designing, configuring, and implementing a sharding solution for large loan and repayment tables using ShardingSphere, Spring Boot 3, MySQL, and MyBatis‑Plus, covering schema planning, data migration, DBA coordination, code examples, common pitfalls, and operational considerations.
This article examines a real‑world MySQL master‑slave replication lag issue caused by massive multi‑table transactions during a sharding operation, details the investigative steps using system tables, binlog parsing, and my2sql analysis, and proposes three practical mitigation strategies.
This article systematically examines MySQL optimization techniques for both read‑many/write‑few and write‑many/read‑few business scenarios, covering architecture design, caching, sharding, asynchronous processing, monitoring, and emerging NewSQL/HTAP solutions to achieve high performance, consistency, and scalability.
Redis cluster relies on two ports per node, a hash‑slot based sharding scheme, a master‑slave replication model, and provides eventual consistency, making these four fundamentals essential knowledge for developers and operators to.
mybatis-mate is an enterprise‑level MyBatis‑Plus extension that offers sharding, data auditing, dictionary binding, field encryption, sensitive‑word filtering, automatic DDL maintenance, multi‑datasource switching, performance logging, and data‑scope control, providing a comprehensive solution for agile and secure database operations.
This article discusses the problem of MySQL auto‑increment ID exhaustion and presents six practical solutions—including changing column type to BIGINT, using UUIDs, segmenting ID generation, composite keys, adjusting auto‑increment steps, and database sharding—to ensure scalability and uniqueness.
This article objectively compares middleware‑based sharding with NewSQL distributed databases, examining their architectural differences, transaction models, high‑availability mechanisms, scaling, SQL support, storage engines, and maturity to help practitioners decide which approach best fits their workload and operational constraints.
Although foreign keys enforce referential integrity, many enterprises avoid them because they can slow inserts, cause blocking cascade updates, trigger database update storms in high‑concurrency environments, and complicate sharding, leading developers to prefer redundant fields and handle relationships at the application layer.
The article explains Elasticsearch indexes as logical namespaces for JSON documents, details how mappings define immutable field types such as text, keyword, and numeric, describes primary and replica shard roles, and offers practical guidance on alias usage, shard sizing, replica settings, and performance optimizations to ensure cluster stability.
This article details the end‑to‑end design, configuration, and code implementation of a sharding solution for loan and repayment tables, covering database schema changes, historical data migration, dynamic synchronization switches, transaction handling, and practical pitfalls when using Spring Boot 3 with ShardingSphere.
The case study details how an activity‑middleware platform tackled slow SQL problems by cleaning historic data, introducing sharding and Elasticsearch off‑loading, optimizing queries with proper indexes and simplified joins, and automating nightly deletions, ultimately cutting daily slow queries from thousands to double‑digit levels and boosting system stability.
This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectures, distributed transaction handling, high‑availability, scaling, storage engines, and ecosystem maturity, and provides guidance on selecting the appropriate approach based on consistency, growth, operational capacity, and performance requirements.
The article objectively compares NewSQL distributed databases with middleware‑plus‑sharding solutions, covering architectural differences, distributed transaction handling, high‑availability, scaling, SQL support, storage engines, maturity, and provides a decision‑making checklist to help engineers select the most suitable approach for their workloads.
This article explains the challenges of storing and serving trillion‑level count data for a massive micro‑blogging platform and presents a step‑by‑step backend design that evolves from simple MySQL tables to sharded databases, Redis caching, message‑queue write‑sharding, and memory‑efficient custom data structures to achieve high performance, low cost, and strong consistency.
This article explains typical sharding strategies—hash, range, and middle‑table—describes SDK, Proxy, and Sidecar middleware forms, and presents several pagination optimization techniques such as global query rewriting, average pagination, disabling cross‑page queries, secondary queries, and using intermediate tables to improve performance in distributed databases.
This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectures, distributed transaction support, high availability, scaling, SQL capabilities, and maturity to help readers decide which approach best fits their workload and operational constraints.