Understanding Xxl-Job: Architecture and Implementation of a Lightweight Distributed Task Scheduler

This article provides a detailed walkthrough of Xxl-Job, a lightweight distributed task scheduling platform, covering its core concepts, demo setup, and internal implementation details.

Core Concepts

The platform consists of three main components: the Scheduling Center (a web service that triggers tasks), the Executor (a service instance that runs the actual job logic), and the Task itself. The scheduling center stores task definitions in a database and can run in cluster mode, sharing data via the same DB.

Demo: Setting Up a Scheduling Center

1. Clone the source code: https://github.com/xuxueli/xxl-job.git

2. Modify the DB connection and execute the SQL files under /doc/db .

3. Start the admin console (default URL http://localhost:8080/xxl-job-admin/toLogin , credentials admin/123456 ).

4. Add an executor named sanyou-xxljob-demo and create a task TestJob with a cron expression that runs every second.

Dependency Configuration (Maven)

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
        <version>2.2.5.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>com.xuxueli</groupId>
        <artifactId>xxl-job-core</artifactId>
        <version>2.4.0</version>
    </dependency>
</dependencies>

Executor Bean Configuration

@Configuration
public class XxlJobConfiguration {
    @Bean
    public XxlJobSpringExecutor xxlJobExecutor() {
        XxlJobSpringExecutor executor = new XxlJobSpringExecutor();
        executor.setAdminAddresses("http://localhost:8080/xxl-job-admin");
        executor.setAppname("sanyou-xxljob-demo");
        executor.setPort(9999);
        executor.setAccessToken("default_token");
        executor.setLogPath("./");
        return executor;
    }
}

Jobs are defined with the @XxlJob("TestJob") annotation:

@Component
public class TestJob {
    private static final Logger logger = LoggerFactory.getLogger(TestJob.class);

    @XxlJob("TestJob")
    public void testJob() {
        logger.info("TestJob任务执行了。。。");
    }
}

Executor Startup Process

When the executor starts, it creates an HTTP server (Netty‑based) on the configured port, registers itself to the scheduling center, and initializes JobHandler instances by scanning beans annotated with @XxlJob . Three JobHandler types exist:

MethodJobHandler – invokes a Java method via reflection.

GlueJobHandler – allows dynamic Java code updates through the GLUE IDE.

ScriptJobHandler – executes scripts (e.g., shell, python).

Task Triggering in the Scheduling Center

The scheduling thread periodically reads the xxl_job_info table, selects tasks whose next fire time is within the next 5 seconds, and classifies them into overdue, near‑due, and future groups. Overdue tasks follow the configured expiration strategy (ignore or fire immediately). After selection, the thread updates the next fire time for each task.

To ensure only one scheduler instance triggers a task, a DB‑based lock is acquired with:

SELECT * FROM xxl_job_lock WHERE lock_name = 'schedule_lock' FOR UPDATE

Fast/Slow Thread Pools

Triggering requests are sent to executors via HTTP. If a task’s trigger latency exceeds 500 ms and this happens more than 10 times within a minute, the task is moved to a slow thread pool to avoid blocking faster tasks.

Executor Routing Strategies

When multiple executor instances exist, the scheduling center selects one based on strategies such as First, Last, Round‑Robin, Random, Least‑Frequently‑Used (LFU), Least‑Recently‑Used (LRU), Failover, Busy‑Transfer, and Sharding‑Broadcast.

Blocking Strategies on Executors

Serial – tasks are queued and executed one after another.

Discard – subsequent triggers are ignored.

Cover – a new thread interrupts the previous one and replaces it.

Result Callback

After execution, the executor places the result into an in‑memory queue. A dedicated TriggerCallbackThread batches these results and posts them back to the scheduling center, which updates task status and handles retries or downstream jobs.

Overall Architecture

The final diagram (from the official site) shows the scheduling center, executors, HTTP communication, and auxiliary components such as logging. Although the original diagram referenced a custom RPC layer, the current implementation uses HTTP for better cross‑language support.