Task Migration | BestHub

DeWu Technology

Jul 31, 2024 · Big Data

Custom Flink Scheduler Enhancements: Resource Balancing, Task Migration, and TmRestart Strategy

The article details Dewu’s custom Flink scheduler, DwScheduler, which adds JSON‑based resource specifications, per‑TaskManager slot sharing for balanced CPU use, hot TaskManager migration callbacks, and a new TmRestart strategy for rapid pod‑process recovery, offering practical techniques to enhance real‑time stream processing stability and performance.

Apache FlinkPerformance OptimizationResource Management

0 likes · 9 min read

Custom Flink Scheduler Enhancements: Resource Balancing, Task Migration, and TmRestart Strategy

OPPO Kernel Craftsman

Apr 3, 2020 · Fundamentals

Linux CPU Load Balancing: Framework and Implementation Analysis

Linux kernel load balancing, explained through the 5.4.28 framework, defines precise CPU load via PELT, balances tasks according to each core’s capacity using hierarchical scheduling domains and groups, details class‑specific strategies, illustrates migration in a big‑little 8‑core system, and outlines the three balancing scenarios and their trigger mechanisms.

CFS SchedulerCPU SchedulerCPU Topology

0 likes · 15 min read

Linux CPU Load Balancing: Framework and Implementation Analysis

58 Tech

Nov 20, 2019 · Big Data

Evolution of 58.com Real-Time Computing Platform and the One‑Stop Streaming Platform Wstream Built on Flink

This article details the technical evolution of 58.com’s real‑time computing platform, describing the shift from Storm and Spark Streaming to Apache Flink, the design of the one‑stop Wstream platform, its large‑scale deployment, stability measures, SQL streaming capabilities, task migration, diagnostics, optimizations, and future plans.

Big DataFlinkReal-time Streaming

0 likes · 11 min read

Evolution of 58.com Real-Time Computing Platform and the One‑Stop Streaming Platform Wstream Built on Flink