PowerJob: A Next‑Generation Distributed Task Scheduling and Computing Framework – Overview and Quick Start Guide

Overview

PowerJob is a next‑generation distributed task scheduling and computing framework that supports CRON, API, fixed‑rate and fixed‑delay scheduling strategies, provides workflow orchestration to handle task dependencies, and enables easy job scheduling and complex distributed computation.

Why Choose PowerJob?

Existing popular job‑scheduling frameworks such as Quartz, elastic‑job and xxl‑job have limitations. Quartz, the first‑generation framework, lacks a web UI, is configured via API only, and only runs on a single node, preventing full cluster utilization.

xxl‑job, considered second‑generation, improves on Quartz but still suffers from issues: it only supports MySQL, has limited distributed computing capability (static sharding only), and does not support workflow/DAG.

In an era of growing data volume and complex business logic, a more powerful scheduler is needed, which led to the creation of PowerJob.

Key Features of PowerJob (Third‑Generation Scheduler)

Ease of Use: Provides a front‑end web UI for visual management of tasks (CRUD), monitoring, and log viewing.

Rich Timing Strategies: Supports CRON expressions, fixed‑rate, fixed‑delay, and API‑driven scheduling.

Multiple Execution Modes: Offers standalone, broadcast, Map, and MapReduce modes; Map/MapReduce processors give cluster‑wide computation with minimal code.

DAG Workflow Support: Allows online configuration of task dependencies and data passing between upstream and downstream tasks.

Broad Executor Support: Executes Spring Bean, built‑in or external Java classes, Shell, Python, etc.

Operations Friendly: Real‑time log display in the web console reduces debugging effort.

Lightweight Dependencies: Only requires a relational database (MySQL/PostgreSQL/Oracle/SQL Server) and works with any Spring Data JPA‑compatible DB.

High Availability & Performance: Lock‑free scheduling design enables horizontal scaling by deploying multiple scheduler servers.

Failover & Retry: Failed tasks can be retried according to configured policies as long as executor nodes are available.

Comparison with Similar Products

Applicable Scenarios

Batch data synchronization at midnight, report generation, cluster‑wide log cleanup, or any workload that benefits from distributed processing such as large‑scale data updates using Map/MapReduce.

Architecture

Quick Start

PowerJob consists of a scheduler server (powerjob‑server) and an executor (powerjob‑worker). The server provides the web service and schedules tasks; the worker runs user‑written task code and offers distributed computing.

Clone the repository:

git clone https://github.com/KFCFans/PowerJob.git

Import the project into an IDE, start the powerjob‑server, and develop processors in the samples module.

Configure the database (e.g., powerjob‑daily) and edit spring.datasource.core.jdbc-url , spring.datasource.core.username , spring.datasource.core.password (or MongoDB URI) in the server’s configuration file.

oms.env=DAILY  
logging.config=classpath:logback-dev.xml  

####### Database configuration #######  
spring.datasource.core.driver-class-name=com.mysql.cj.jdbc.Driver  
spring.datasource.core.jdbc-url=jdbc:mysql://remotehost:3306/powerjob-daily?useUnicode=true&characterEncoding=UTF-8  
spring.datasource.core.username=root  
spring.datasource.core.password=No1Bug2Please3!  
spring.datasource.core.hikari.maximum-pool-size=20  
spring.datasource.core.hikari.minimum-idle=5  

####### MongoDB configuration (optional) #######  
spring.data.mongodb.uri=mongodb://remotehost:27017/powerjob-daily  

####### Mail configuration (if needed) #######  
spring.mail.host=smtp.163.com  
spring.mail.username=zqq  
spring.mail.password=qqz  
spring.mail.properties.mail.smtp.auth=true  
spring.mail.properties.mail.smtp.starttls.enable=true  
spring.mail.properties.mail.smtp.starttls.required=true  

####### Log retention configuration #######  
oms.log.retention.local=1  
oms.log.retention.remote=1  
oms.container.retention.local=1  
oms.container.retention.remote=-1  
oms.instanceinfo.retention=1  

####### Cache configuration #######  
oms.instance.metadata.cache.size=1024

Start the server with the main class com.github.kfcfans.powerjob.server.OhMyApplication and verify the web UI at http://127.0.0.1:7700/ . Register an application (e.g., oms-test ) via the UI.

Write a Sample Processor

In the powerjob-worker-samples module, adjust the configuration to point to the registered app name, then create a processor class that implements BasicProcessor :

@Slf4j  
@Component  
public class StandaloneProcessorDemo implements BasicProcessor {  

    @Override  
    public ProcessResult process(TaskContext context) throws Exception {  

        OmsLogger omsLogger = context.getOmsLogger();  
        omsLogger.info("StandaloneProcessorDemo start process,context is {}.", context);  
        System.out.println("jobParams is " + context.getJobParams());  

        return new ProcessResult(true, "process successfully~");  
    }  
}

Run the sample application com.github.kfcfans.powerjob.samples.SampleApplication and confirm successful startup.

Task Configuration and Execution

After both server and sample project are running, open the web console, select the registered app, create a new task, and either wait for the schedule or click “Run” to execute immediately. Monitor task status and logs from the UI.

Further Resources

Project repository: https://github.com/KFCFans/PowerJob

Official documentation: https://www.yuque.com/powerjob/guidence/ztn4i5

Online trial: https://www.yuque.com/powerjob/guidence/hnbskn