Arctic on Flink: Streaming Features, Core Principles, Benchmark Results, and Future Roadmap

Introduction – The session introduces Arctic on Flink, focusing on its streaming lakehouse service, mixed‑format support, and the four main topics: streaming features, core principles, benchmark, and future planning.

Arctic Streaming Features – Describes Arctic's architecture, including Streaming Lakehouse Service, support for Iceberg and upcoming formats, and mixed‑format capabilities such as millisecond‑level data processing, primary‑key based upserts, minute‑level OLAP timeliness, and additional features like efficient dimension‑table joins.

Core Principles – Explains the three‑layer table structure (Change Store, Base Store, Log Store), how Flink CDC integrates with Arctic, the handling of updates via before/after records, optimizer tasks for file merging, and the hybrid query model that reads both base and change data.

Benchmark – Presents performance tests for static and dynamic data workloads, comparing Arctic with Iceberg on OLAP queries, highlighting Arctic's optimizer advantages in dynamic scenarios and its ability to handle high‑throughput workloads using Kafka/Pulsar connectors.

Future Planning – Outlines upcoming work such as partial‑field upserts, unified Logstore/Filestore reads using FLIP‑27 APIs, dimension‑table join optimizations, and improvements to initialization and data‑skew handling.

Q&A – Provides answers to common questions about model support, data migration, tree structures, join performance, Flink vs. Trino query capabilities, and roadmap items.

Resources – Links to the Arctic GitHub repository, documentation site, and community contact information.