distributed scheduling

Java Architect Essentials

Jun 10, 2025 · Backend Development

Why Rebuild a Distributed Scheduler? Inside a Custom Java Job Framework

This article explains the motivations behind creating a new distributed scheduling framework, compares existing solutions, and details the design choices—including gRPC communication, protobuf serialization, a custom NameServer for load balancing, and a built‑in message queue with persistence and retry mechanisms—to handle frequent task creation and dynamic parameter changes in a high‑concurrency environment.

JavaMessage Queuedistributed scheduling

0 likes · 14 min read

Why Rebuild a Distributed Scheduler? Inside a Custom Java Job Framework

Alibaba Cloud Infrastructure

May 26, 2025 · Big Data

Comparative Guide to Apache Airflow and Argo Workflows for Distributed Task Scheduling

This article provides a comprehensive comparison of Apache Airflow and Argo Workflows, covering their core features, architectures, use cases, code examples, and recommendations for selecting the appropriate distributed workflow engine in data engineering, big‑data, and AI pipelines.

Apache AirflowArgo WorkflowsBig Data

0 likes · 23 min read

Comparative Guide to Apache Airflow and Argo Workflows for Distributed Task Scheduling

Xiaohongshu Tech REDtech

May 22, 2025 · Artificial Intelligence

Scalable Overload-Aware Graph-Based Index Construction for 10‑Billion‑Scale Vector Similarity Search (SOGAIC)

The paper introduces SOGAIC, a scalable overload‑aware graph‑based index construction system for billion‑scale vector similarity search that uses adaptive overlapping partitioning and load‑balanced distributed scheduling to cut construction time by 47.3% while maintaining high recall.

ANNLarge ScaleVector Search

0 likes · 13 min read

Scalable Overload-Aware Graph-Based Index Construction for 10‑Billion‑Scale Vector Similarity Search (SOGAIC)

macrozheng

May 14, 2025 · Backend Development

Why PowerJob Is the Next‑Gen Distributed Scheduler You Should Try

This article introduces PowerJob, a next‑generation distributed scheduling and computing framework, outlines its key advantages, compares it with other schedulers, and provides a step‑by‑step guide for local deployment, web console usage, and creating a scheduled task.

PowerJobSpring Bootdistributed scheduling

0 likes · 6 min read

Why PowerJob Is the Next‑Gen Distributed Scheduler You Should Try

Top Architect

Apr 18, 2025 · Backend Development

Comprehensive Guide to Using the XXL‑Job Distributed Task Scheduling Framework

This article provides a step‑by‑step tutorial on installing, configuring, and using the open‑source XXL‑Job distributed scheduling framework with Spring Boot, covering server and executor deployment, task creation via annotations, API and sharding, execution, logging, and the underlying Netty‑based communication design.

Backend DevelopmentJavaSpring Boot

0 likes · 14 min read

Comprehensive Guide to Using the XXL‑Job Distributed Task Scheduling Framework

Code Ape Tech Column

Feb 18, 2025 · Backend Development

Practical Guide to Using XXL‑JOB for Distributed Task Scheduling with Spring Boot and Docker

This article explains what XXL‑JOB is, why a distributed task‑scheduling platform is needed, and provides a step‑by‑step tutorial—including Docker deployment, Spring Boot integration, sharding, idempotency handling, and a video‑transcoding use case—to help developers implement reliable distributed jobs in Java applications.

DockerJavaSpring Boot

0 likes · 19 min read

Practical Guide to Using XXL‑JOB for Distributed Task Scheduling with Spring Boot and Docker

Top Architect

Jan 16, 2025 · Backend Development

Guide to Using the XXL‑JOB Distributed Task Scheduling Framework with Spring Boot

This article introduces the XXL‑JOB open‑source distributed scheduler, compares it with Quartz and other solutions, and provides a step‑by‑step tutorial for server deployment, executor configuration, task development, execution, logging, and the underlying Netty‑based communication design, complete with code examples.

JavaNettySpring Boot

0 likes · 15 min read

Guide to Using the XXL‑JOB Distributed Task Scheduling Framework with Spring Boot

macrozheng

Sep 25, 2024 · Backend Development

Master Distributed Task Scheduling with XXL-JOB: Docker Setup to Spring Boot Integration

This comprehensive guide explains what XXL-JOB is, why distributed task scheduling is needed, and walks through installing XXL-JOB with Docker, configuring it in a Spring Boot project, creating custom jobs, handling sharding, ensuring idempotency, and monitoring execution, all illustrated with code and diagrams.

DockerMicroservicesSpring Boot

0 likes · 21 min read

Master Distributed Task Scheduling with XXL-JOB: Docker Setup to Spring Boot Integration

Top Architect

Sep 7, 2024 · Backend Development

XXL-Job Distributed Task Scheduling Framework: Project Overview and Hands‑On Guide

This article introduces the open‑source XXL-Job distributed task scheduling framework, explains its architecture and communication design, and provides a step‑by‑step hands‑on tutorial covering server deployment, executor configuration, task development (annotation, API, sharding), execution, and log inspection, with code examples and screenshots.

Backend DevelopmentJavaSpring Boot

0 likes · 14 min read

XXL-Job Distributed Task Scheduling Framework: Project Overview and Hands‑On Guide

JD Tech Talk

Jul 25, 2024 · Backend Development

Design and Architecture of JD.com’s Buffalo Distributed DAG Scheduling System

The article details the design, core technical solutions, high‑availability architecture, performance optimizations, and open capabilities of Buffalo, JD.com’s distributed DAG‑based job scheduling platform that supports massive task volumes, complex dependencies, and flexible resource management.

DAGOperationsbackend

0 likes · 13 min read

Design and Architecture of JD.com’s Buffalo Distributed DAG Scheduling System

JD Retail Technology

Jul 22, 2024 · Big Data

Design and Architecture of JD's Buffalo Distributed Workflow Scheduling System

The article introduces JD's Buffalo distributed workflow scheduling system, detailing its dual-layer entity model, instance-based scheduling, high‑availability three‑tier architecture, performance optimizations such as horizontal scaling and event‑driven execution, as well as cold‑hot data separation and open APIs for future enhancements.

Big DataBuffaloJD

0 likes · 10 min read

Design and Architecture of JD's Buffalo Distributed Workflow Scheduling System

Selected Java Interview Questions

Jun 19, 2024 · Backend Development

PowerJob: Introduction, Installation, and Task Configuration Guide

This article introduces the PowerJob Java distributed scheduling framework, explains its features and task types, provides step‑by‑step installation via JAR and Docker, demonstrates database setup, server and worker configuration, and details how to create and configure scheduled tasks with code examples.

JavaPowerJobbackend

0 likes · 11 min read

PowerJob: Introduction, Installation, and Task Configuration Guide

Code Ape Tech Column

Jun 16, 2024 · Backend Development

Introducing PowerJob: A Lightweight Distributed Task Scheduling Framework and Its Usage

This article introduces PowerJob, a young yet mature distributed task scheduling framework, explains why it was chosen, details its architecture, high‑availability design, deployment steps, and demonstrates various job types—including standalone, broadcast, map, and MapReduce—along with CRON, fixed‑rate, and fixed‑delay scheduling configurations.

JavaMapReducePowerJob

0 likes · 13 min read

Introducing PowerJob: A Lightweight Distributed Task Scheduling Framework and Its Usage

DataFunTalk

May 23, 2024 · Big Data

Berserker Big Data Platform: Architecture, Development Practices, and Operational Enhancements

This article presents a comprehensive overview of the Berserker big‑data platform, detailing its overall design, data‑development components, key architectural challenges such as state management, release processes, two‑phase commit, RPC duplication, task routing, message handling, execution isolation, dependency model redesign, and outlines future work including stateless execution nodes, Kubernetes integration, and unified stream‑batch processing.

Big DataData PlatformDocker

0 likes · 15 min read

Berserker Big Data Platform: Architecture, Development Practices, and Operational Enhancements

Java Tech Enthusiast

Feb 14, 2024 · Backend Development

Understanding Xxl-Job: Architecture and Implementation of a Lightweight Distributed Task Scheduler

The article explains Xxl‑Job’s lightweight distributed scheduling architecture—detailing its Scheduling Center, Executor, and Task components, demo installation steps, Maven configuration, executor bean setup, job‑handler types, scheduling logic, thread‑pool and routing strategies, blocking policies, result callbacks, and overall HTTP‑based communication flow.

JavaTask ExecutionXxl-Job

0 likes · 19 min read

Understanding Xxl-Job: Architecture and Implementation of a Lightweight Distributed Task Scheduler

Code Ape Tech Column

Feb 11, 2024 · Backend Development

Comparison of Java Distributed Scheduling Frameworks: Elastic-Job vs X-Job and Other Options

This article examines why scheduled tasks are needed, outlines common Java scheduling frameworks for both single‑machine and distributed environments, and provides a detailed side‑by‑side comparison of Elastic‑Job, X‑Job, Quartz and other solutions, highlighting their architectures, clustering, sharding, monitoring, scaling and failure‑handling strategies.

Javabackenddistributed scheduling

0 likes · 13 min read

Comparison of Java Distributed Scheduling Frameworks: Elastic-Job vs X-Job and Other Options

Java Captain

Feb 4, 2024 · Backend Development

Integrating XXL-JOB with SpringBoot for Distributed Task Scheduling

This article provides a step‑by‑step guide on integrating the lightweight distributed task scheduler XXL‑JOB into a SpringBoot application, covering prerequisite installations, Maven dependency configuration, property settings, JobHandler implementation, and registration to enable scalable and reliable job execution.

Backend DevelopmentJavaSpringBoot

0 likes · 3 min read

Integrating XXL-JOB with SpringBoot for Distributed Task Scheduling

macrozheng

Dec 15, 2023 · Backend Development

How Xxl-Job Powers Lightweight Distributed Task Scheduling in Java

This article explains the core concepts, architecture, and implementation details of the lightweight distributed scheduling platform Xxl-Job, including its scheduler center, executor, job handlers, registration, task triggering, routing strategies, execution flow, and result callbacks, with a step‑by‑step demo and code examples.

JavaSpringBootbackend

0 likes · 21 min read

How Xxl-Job Powers Lightweight Distributed Task Scheduling in Java

DaTaobao Tech

Dec 11, 2023 · Big Data

Design and Implementation of an Online Batch Processing Framework for Large-Scale Promotion Systems

The paper presents a centralized online batch‑processing framework for large‑scale promotion systems, where applications integrate via an SDK, a task‑center schedules and dispatches sub‑tasks through RocketMQ to Dubbo‑enabled containers, employing MapReduce‑style splitting, Guava rate‑limiting, heartbeat health checks, and has successfully handled over 1.3 million tasks during Double‑11.

Big DataDubboJava

0 likes · 9 min read

Design and Implementation of an Online Batch Processing Framework for Large-Scale Promotion Systems

Code Ape Tech Column

Dec 9, 2023 · Backend Development

PowerJob Overview: Selection Rationale, Architecture, Task Types, and Scheduling Strategies with Code Samples

This article introduces the PowerJob distributed task framework, explains why it was chosen, details its architecture and high‑availability design, demonstrates various job types—including standalone, broadcast, map, and map‑reduce—with Java code examples, and covers scheduling options such as CRON, fixed‑rate, and fixed‑delay execution.

JavaMapReducePowerJob

0 likes · 14 min read

PowerJob Overview: Selection Rationale, Architecture, Task Types, and Scheduling Strategies with Code Samples