Apache Hudi Concurrency Control: Overview, MVCC, and OCC

Overview – Every commit to a Hudi table forms a transaction, whether it adds new data or runs a table service job. Concurrency control coordinates simultaneous transactions to ensure correctness, consistency, and high performance.

ACID recap – Atomicity, Consistency, Isolation, and Durability are briefly described to illustrate why isolation is essential for preventing dirty reads, lost updates, and other anomalies.

MVCC in Hudi – Hudi uses Multi‑Version Concurrency Control (MVCC) by maintaining a timeline of monotonically increasing commit timestamps and file slices that represent record versions. The TableFileSystemView API provides the latest view of the table, enabling writers and readers to operate without locking and ensuring read‑write isolation.

OCC in Hudi – Optimistic Concurrency Control (OCC) follows a three‑phase protocol: read, validate, and write. Hudi implements OCC at the file‑level to support multiple writers. Users enable it by setting hoodie.write.concurrency.mode=OPTIMISTIC_CONCURRENCY_CONTROL and configuring a lock provider (e.g., Zookeeper, Hive Metastore, DynamoDB).

The article illustrates OCC with a detailed example of two write clients acquiring locks, checking the timeline, and handling conflicts, showing how conflicting writes are aborted and rolled back to preserve atomicity.

Summary – The article recaps the importance of MVCC and OCC for handling single‑writer/table‑service and multi‑writer scenarios in Hudi, and invites readers to join the Apache Hudi community for further discussion.