1 views collected around this technical thread.
The article outlines why deleting tens of millions of rows in one statement harms performance and reliability, then presents safe, efficient tactics—such as indexed batch deletes, partition drops, creating a new table, or using TRUNCATE—plus pre‑deletion planning, post‑delete validation, monitoring, and space reclamation.
This article explains the risks of bulk deletions on million‑row tables and provides a step‑by‑step guide—including pre‑deletion planning, indexing, backup, batch deletion, partitioning, table recreation, TRUNCATE, and post‑deletion verification—to safely and quickly remove large volumes of data while minimizing lock contention, log growth, CPU spikes, replication lag, and rollback complications.
This article explains how adding an index to a status column in a tens‑of‑millions‑row MySQL table affects query performance, covering the basic index lookup process, extreme cases where full scans are chosen, selectivity, covering indexes, composite indexes, partitioning, and using EXPLAIN to verify execution plans.
This article explains how to assess MySQL table size using table capacity, disk usage, and instance resources, describes why large tables degrade performance, and presents three practical solutions—partitioning, sharding (horizontal/vertical), and hot‑cold data archiving—along with their trade‑offs and implementation tips.
This article explains how Hive serves as a data‑warehouse layer for user‑profile tagging, covering data‑warehouse fundamentals, fact‑and‑dimension modeling, partitioned storage, label aggregation, and ID‑mapping techniques with practical Hive DDL/DML examples.
This article examines the replication and partitioning designs of Redis, Kafka, and Elasticsearch, drawing on concepts from Designing Data‑Intensive Applications to illustrate core distributed‑system principles, common challenges, and practical configuration options.
This article explains the fundamentals of database sharding and partitioning, why they are needed for massive data volumes, various vertical and horizontal splitting strategies, routing algorithms, client vs. proxy deployment models, and the practical challenges and solutions when using ShardingSphere 5.x.
This article explains how to assess MySQL table size through table capacity, disk usage, and instance resources, illustrates the impact of B+‑tree index depth on query speed, and presents three practical solutions—table partitioning, database sharding, and hot‑cold data archiving—to improve performance.
This article explains the concepts of local and global indexes in OceanBase, demonstrates how to create tables and indexes, compares six practical scenarios—including no index, local index, partitioned queries, unique and global indexes—and provides performance metrics to guide database optimization.
This article explains how to design efficient StarRocks tables with proper field types, partitioning and bucketing, compares update and primary‑key models, presents real‑world cases of memory and tablet issues, provides a complete table‑creation example, and outlines comprehensive monitoring metrics to keep the analytical data warehouse performant and stable.
This article provides a comprehensive overview of Kafka, covering its core architecture, message queue models, communication process, partition selection, consumer groups, rebalancing strategies, partition assignment algorithms, reliability guarantees, replica synchronization, and reasons for removing Zookeeper in newer versions.
This article explains how Apache Doris implements data sharding, partitioning (both list and range, static and dynamic), batch partition creation, and the Auto Bucket feature, providing detailed syntax, algorithms, and performance benefits for large‑scale distributed databases.
This article presents the background, overall architecture, partitioning strategies, raft‑based replication, binlog support, multi‑active deployment, bulk‑load mechanisms, storage‑engine optimizations, load‑balancing policies, and failure‑detection & recovery techniques of a high‑reliability, high‑throughput key‑value store used at Bilibili.
This article presents a practical guide to implementing monthly sharding for high‑volume billing data using MySQL stored procedures, dynamic table creation, and SpringBoot‑MyBatis integration, covering schema design, code examples, and query strategies for inserting, retrieving, and managing partitioned tables.
This article explains how to classify massive order data into hot and cold segments, store them in MySQL, Elasticsearch and Hive respectively, and implement sharding and partitioning at both table and database levels using modulo and hash calculations to achieve scalable performance for billions of orders.
The article examines MySQL partition tables that lack primary keys, compares query performance against tables with primary keys, and provides guidance on index design for such partitioned tables, concluding that partition keys need not be included in indexes.
This article details the design and implementation of a high‑availability, high‑performance membership system that serves billions of users across multiple platforms, covering Elasticsearch dual‑center clusters, traffic‑isolated three‑cluster setups, Redis caching strategies, MySQL dual‑center partitioning, and advanced flow‑control and degradation mechanisms.
This article explains how to choose storage options, design sharding keys, apply partitioning strategies, modify business code, handle historical data migration, and plan future scaling for large‑scale e‑commerce systems using MySQL, NoSQL, and NewSQL solutions.
This article provides a comprehensive overview of Kafka's architecture, covering topics, partitions, replication, producer and consumer workflows, offset management, rebalancing, delivery semantics, exactly‑once guarantees, transaction handling, file organization, and key configuration settings.
This comprehensive guide covers Kafka’s core concepts, architecture, Zookeeper role, producer sending modes, partitioning strategies, replica types, message deletion, performance optimizations, consumer models, offset management, and best‑practice recommendations for scaling and ensuring ordered delivery in distributed streaming systems.