1 views collected around this technical thread.
The article reviews Apache Flink's rapid development over the past decade, highlighting its 2023 community growth, SIGMOD award, major releases, streaming SQL enhancements, incremental checkpointing, batch maturity, cloud‑native scaling, and integration with the emerging Lakehouse architecture.
This article explains how to enable and use the OpenJDK CRaC project with SpringBoot 3.2, covering prerequisites, dependency setup, automatic and manual checkpoint creation, and demonstrates significant startup‑time reductions through detailed examples and performance results.
This article demonstrates how to enable and use Coordinated Restore at Checkpoint (CRaC) in Spring Boot 3.2 with the Spring Petclinic example, covering required dependencies, JVM setup, automatic and manual checkpoint creation, performance measurements, and best‑practice considerations for backend developers.
This article demonstrates how Spring Boot 3.2 leverages the OpenJDK CRaC project to dramatically cut application startup time, covering required JVM, dependencies, Zulu JDK setup, automatic and manual checkpoint creation, performance benchmarks, and restoration steps, all using the Petclinic example.
An incident report details how a scheduled machine reboot on Alibaba Cloud triggered a Flink TaskManager failover, leading to excessive data replay, increased ES pressure, and significant business latency, and explains the root cause involving disabled checkpoints and timestamp‑based offset consumption.
This article explains the Asynchronous Barrier Snapshotting algorithm used by Apache Flink for checkpointing, detailing its origins from the Chandy‑Lamport algorithm, its operation in both acyclic and cyclic dataflow graphs, barrier alignment, and the fault‑recovery process.
This article presents a detailed guide for optimizing large‑scale Apache Flink streaming jobs on the JD Real‑Time Computing platform, covering TaskManager memory model tuning, network stack configuration, RocksDB state management, checkpoint strategies, and additional performance tips with practical examples and calculations.
The article explains how Spark Streaming's checkpoint mechanism works, detailing the four-step process—from setting the checkpoint directory to writing RDD data and finalizing the checkpoint—highlighting its role in ensuring fault‑tolerant, fast recovery for real‑time recommendation feature pipelines.
This article explains how Apache Flink achieves end‑to‑end exactly‑once semantics by using source replay support, checkpoint‑based snapshots, asynchronous incremental checkpoints, and two‑phase commit sinks, and describes the interaction with external systems such as Kafka to ensure transactional writes.
This article provides an extensive overview of Apache Flink, covering deployment modes, cluster sizing, job submission workflows, state programming concepts, checkpoint mechanisms, backpressure handling, comparison with Spark, and practical code snippets for configuration and optimization.
This article provides a comprehensive overview of MySQL, detailing its configuration file search order, component architecture, various storage engines such as MyISAM, NDB, Memory, and an in‑depth examination of InnoDB’s internal structures, memory management, background threads, LRU handling, redo log buffering, and checkpoint mechanisms.
The article systematically introduces Apache Flink’s streaming computation model, contrasting batch and real‑time processing, detailing its unified architecture, managed and raw state with key groups, checkpointing and savepoints for fault tolerance, data exchange mechanisms, time semantics, windowing, side‑outputs, and a complete Java Kafka‑based example.
This article details Bilibili's comprehensive optimization of Flink real‑time computing, covering connector stability improvements, SQL interval‑join enhancements, runtime state and checkpoint refinements, a diagnostic tool, and future directions for high‑throughput streaming workloads.
This article explains the meaning of the 'Pages flushed up to' value in MySQL's InnoDB engine, how it differs from the 'Last checkpoint' LSN, the underlying logic for its calculation, and provides test results demonstrating its behavior during writes and checkpoints.
This article explains the internal mechanisms of relational databases, covering data pages, buffer pools, undo and redo logs, checkpointing, and how these components work together to provide atomicity, durability, and crash recovery while minimizing disk I/O.
This article reviews the Pravega project and its Flink connector, covering Pravega's design for large‑scale streaming, the connector's evolution and exact‑once semantics, Flink 1.11 integration challenges, checkpoint mechanisms, and future plans such as schema‑registry and new Flink features.
This article presents ByteDance's recent Flink enhancements, detailing a single‑point recovery mechanism for the network layer and a regional checkpoint strategy that together improve failover latency, reduce output loss, and enable scalable, high‑throughput stream processing for large‑scale real‑time recommendation workloads.
Burst Buffer is a storage acceleration technology that enhances I/O bandwidth and OPS for high‑performance computing by providing fast checkpoint/restart, temporary storage, and balancing SSD and parallel file system resources, with implementations from DDN, Cray, EMC, and IBM detailed for HPC designers.
This article provides a detailed examination of Elasticsearch's distributed consistency mechanisms, covering the shard write path, the PacificA replication algorithm, the role of SequenceNumber and Checkpoint, and a comparison of ES's implementation with the original algorithm, based on version 6.2.
The article thoroughly explains Apache Flink’s checkpoint mechanism—including state types, coordinator workflow, exactly‑once versus at‑least‑once semantics, common failure sources such as code exceptions, storage or network issues, and practical configuration tips like interval settings, local recovery and externalized checkpoints.