Core Features and Architecture of ClickHouse

ClickHouse is the columnar, massively parallel processing (MPP) database that powers Yandex.Metrica, handling over 200 billion tracking events daily and storing more than 20 trillion rows with 90% of custom queries returning in under one second.

Core DBMS features include full DDL/DML support, fine‑grained permission control, backup/restore, and automatic cluster management, making ClickHouse a true Database Management System rather than just a storage engine.

Columnar storage and compression reduce I/O by scanning only required columns and applying LZ4 compression (average 8:1 ratio in production), which also enables vectorized execution.

Vectorized execution engine leverages SIMD (SSE4.2) instructions to process data in registers, dramatically speeding up operations compared to row‑by‑row loops.

Relational model and SQL provide standard database concepts (databases, tables, views, functions) and a case‑sensitive SQL dialect supporting GROUP BY, ORDER BY, JOIN, IN, etc., facilitating integration with existing analytics tools.

Table engines are abstracted via the IStorage interface; ClickHouse ships with more than 20 engines (MergeTree, TinyLog, Kafka, etc.) allowing users to choose the best fit for their workload.

Multithreading and distributed processing combine data‑level parallelism (vectorization) with thread‑level parallelism, using shards and replicas to scale horizontally while keeping data close to the CPU.

Multi‑master architecture gives every node equal responsibility, eliminating single points of failure and simplifying multi‑data‑center deployments.

Online query performance rivals commercial solutions (Vertica, SparkSQL, Elasticsearch) while remaining open source, delivering sub‑second responses even on complex analytical queries.

Architecture design includes:

Column and Field objects representing columnar data and individual values.

DataType handling serialization/deserialization for various formats.

Block and Block streams that encapsulate columns, types, and names for data flow.

Table abstraction via IStorage, with each engine implementing its own read/write logic.

Parser creates an AST from SQL; Interpreter executes the AST, building pipelines of Block streams.

Functions (ordinary and aggregate) operate on whole columns using vectorized code.

Cluster and Replication model: shards are logical groups, replicas are physical copies; a node hosts exactly one shard.

Example cluster configuration (code list 1):

shard:
  - internal_replication: true
    replicas:
      - host: 10.37.129.6
        port: 9000

To achieve true 1‑shard‑1‑replica semantics, a second replica must be added (code list 2).

Performance secrets stem from a bottom‑up design philosophy: hardware‑first thinking, algorithmic choice over abstraction, aggressive use of SIMD, scenario‑specific optimizations (e.g., different uniq algorithms), code generation for loop unrolling, continuous real‑data testing, and rapid monthly releases.

In summary, ClickHouse’s speed results from the combination of columnar storage, vectorized execution, flexible table engines, multi‑master clustering, and a relentless focus on low‑level performance engineering.