1 views collected around this technical thread.
This article explains how moving aggregation logic from in‑memory processing to ClickHouse SQL, synchronizing configuration data, and leveraging ClickHouse ReplacingMergeTree tables with local joins dramatically reduces query latency and memory consumption for large‑scale supply‑chain planning workloads.
The talk explains how Abase2 tackles multi‑region NoSQL deployment challenges by integrating CRDT technology—compatible with Redis semantics—to achieve conflict‑free, low‑latency data replication, describing its three‑layer architecture, mixed‑logic clocks, cache‑driven log engine, and practical engineering optimizations.
ByteDance’s article details the high‑availability challenges of ClickHouse in large‑scale deployments—such as frequent failures, long recovery times, and operational complexity—and explains three key enhancements: a new HaMergeTree engine to lessen Zookeeper load, RocksDB‑based metadata persistence for faster restarts, and additional reliability features like HaKafka and monitoring tools.
Tencent’s MongoDB team solved the severe performance degradation caused by millions of collections by redesigning WiredTiger to use a shared table‑space with prefix‑mapped keys, eliminating data‑handle explosion, cutting memory use and lock contention, reducing startup from hours to minutes, and boosting read/write throughput by up to two orders of magnitude while preserving full MongoDB compatibility.
The article examines the difficulties of developing and sustaining a multi‑million‑line C‑language database, highlighting complex flag‑driven logic, extensive testing pipelines, slow iteration cycles, and the need for automation, modularity, and robust engineering practices to ensure code quality at scale.