Apache Hudi from Zero to One: Highlighting Key Features of Version 1.0 (Part 10)

This article, translated from the original English blog, introduces Apache Hudi’s architecture hierarchy and highlights four major new features in the upcoming 1.0 release.

Hudi Architecture Layers : The system consists of three layers—transaction database, programming API, and user interface—running on storage systems and providing a transactional database experience for lakehouses.

1.0 Highlights :

LSM‑tree Timeline : Uses a log‑structured merge tree to store an almost unlimited timeline of immutable transaction logs, balancing storage optimization and efficient access.

Non‑blocking Concurrency Control (NBCC) : Introduces MVCC‑based NBCC for MOR tables, allowing multiple writers to update the same file slice without blocking and deferring conflict resolution to the compaction phase.

File‑group Reader and Writer APIs : Leverages record keys and the file‑group model to enable partial updates and position‑based merges, reducing log file size and improving snapshot queries.

Function Indexes : Adds column‑level function indexes (e.g., hour‑based index on a timestamp column) that accelerate data skipping without requiring physical partitioning.

The article concludes with a brief review and points readers to the Hudi 1.x RFC for a concise summary of the upgrade.