Backend Development 14 min read

Integrating XXL‑Job for Scheduled Hot‑Search Crawlers in a Java Backend

This tutorial explains how to replace the basic @Scheduled annotation with the flexible XXL‑Job distributed scheduler, covering repository download, admin deployment, database initialization, Spring‑Boot executor configuration, job registration for Douyin and Bilibili hot‑search crawling, and a Vue front‑end component for displaying ranked results with real‑time update timestamps.

Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Integrating XXL‑Job for Scheduled Hot‑Search Crawlers in a Java Backend

We have built a complete hot‑search component and now replace the simple @Scheduled annotation with the more flexible XXL‑Job distributed task scheduler.

First, download the XXL‑Job repository (≈27.3k stars) and configure the admin center, including JDBC URL, mail alarm, token, i18n, thread pool and log retention settings.

Initialize the database tables using the provided SQL file in the db directory.

Start the admin console by running XxlJobAdminApplication and log in with the default credentials (admin / 123456).

Create a Spring‑Boot executor by adding the xxl-job-core dependency to pom.xml and defining XxlJobConfig with properties such as adminAddresses , appname , port , etc.

Register a job using the @XxlJob("douyinHotSearchJob") annotation; the job fetches Douyin hot‑search data via OkHttp, builds SbmyHotSearchDO objects, stores them in a cache, and persists them to the database.

Similarly, a Bilibili hot‑search job is implemented with @XxlJob("bilibiliHotSearchJob") , extracting fields such as title, author, cover, and view count.

Finally, a Vue component displays the hot‑search list with ranking, heat formatting, and real‑time update timestamps.

Javabackend developmentSpring Bootxxl-jobweb crawlingscheduled tasks
Rare Earth Juejin Tech Community
Written by

Rare Earth Juejin Tech Community

Juejin, a tech community that helps developers grow.

0 followers
Reader feedback

How this landed with the community

login Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.