Design and Implementation of a Generic Asynchronous Processing SDK for Spring Applications

Preface

A good system design must obey the open‑closed principle; as business evolves, core code changes increase the probability of errors. Most added functionality extends existing features, requiring both performance and quality, so we often use asynchronous thread pools, which introduce many uncertainties. To address this, I designed a generic asynchronous processing SDK that can easily implement various async tasks.

Purpose

Asynchronous processing ensures that methods are executed effectively without blocking the main flow.

More importantly, various fallback mechanisms guarantee that data is not lost, achieving eventual consistency.

Advantages

Non‑intrusive design with independent database, scheduled tasks, message queue, and manual execution UI (single sign‑on authentication).

Uses Spring transaction event mechanism; even if async strategy parsing fails, business logic is unaffected.

If a method runs within a transaction, the event is processed after the transaction commits or rolls back.

Even after transaction commit, if async strategy parsing fails, a fallback still executes (unless DB, MQ, or the method itself fails).

Principle

After container initialization, all beans are scanned and methods annotated with @AsyncExec are cached.

When a method runs, an AOP aspect publishes an event.

The transaction event listener processes the asynchronous execution strategy.

@TransactionalEventListener(fallbackExecution = true, phase = TransactionPhase.AFTER_COMPLETION)

fallbackExecution=true processes the event even when no transaction is active.

TransactionPhase.AFTER_COMPLETION handles the event after transaction commit or rollback.

Components

Kafka message queue

XXL‑Job scheduled tasks

MySQL database

Spring AOP aspect

Vue UI

Design Patterns

Strategy

Template Method

Dynamic Proxy

Flowchart

Database Scripts

CREATE TABLE `async_scene` (
  `id` bigint NOT NULL AUTO_INCREMENT COMMENT 'Primary Key',
  `application_name` varchar(100) NOT NULL DEFAULT '' COMMENT 'Application Name',
  `method_sign` varchar(50) NOT NULL DEFAULT '' COMMENT 'Method Signature',
  `scene_name` varchar(200) NOT NULL DEFAULT '' COMMENT 'Business Scene Description',
  `async_type` varchar(50) NOT NULL DEFAULT '' COMMENT 'Async Strategy Type',
  `queue_name` varchar(200) NOT NULL DEFAULT '' COMMENT 'Queue Name',
  `theme_value` varchar(100) NOT NULL DEFAULT '' COMMENT 'Consumer Topic',
  `exec_count` int NOT NULL DEFAULT '0' COMMENT 'Retry Count',
  `exec_deleted` int NOT NULL DEFAULT '0' COMMENT 'Delete After Execution',
  `async_version` varchar(50) NOT NULL DEFAULT '' COMMENT 'Component Version',
  `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Creation Time',
  `update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Update Time',
  `cdc_crt_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Record Insert Time',
  `cdc_upd_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT 'Record Update Time',
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE KEY `uk_application_sign` (`application_name`,`method_sign`) USING BTREE,
  KEY `idx_cdc_upd_time` (`cdc_upd_time`)
) ENGINE=InnoDB COMMENT='Async Scene Table';

CREATE TABLE `async_req` (
  `id` bigint NOT NULL AUTO_INCREMENT COMMENT 'Primary Key',
  `application_name` varchar(100) NOT NULL DEFAULT '' COMMENT 'Application Name',
  `sign` varchar(50) NOT NULL DEFAULT '' COMMENT 'Method Signature',
  `class_name` varchar(200) NOT NULL DEFAULT '' COMMENT 'Full Class Name',
  `method_name` varchar(100) NOT NULL DEFAULT '' COMMENT 'Method Name',
  `async_type` varchar(50) NOT NULL DEFAULT '' COMMENT 'Async Strategy Type',
  `exec_status` tinyint NOT NULL DEFAULT '0' COMMENT 'Execution Status 0:Init 1:Fail 2:Success',
  `exec_count` int NOT NULL DEFAULT '0' COMMENT 'Execution Count',
  `param_json` longtext COMMENT 'Request Parameters',
  `remark` varchar(200) NOT NULL DEFAULT '' COMMENT 'Business Description',
  `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Creation Time',
  `update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Update Time',
  PRIMARY KEY (`id`) USING BTREE,
  KEY `idx_applocation_name` (`application_name`) USING BTREE,
  KEY `idx_exec_status` (`exec_status`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Async Request Table';

CREATE TABLE `async_log` (
  `id` bigint NOT NULL AUTO_INCREMENT COMMENT 'Primary Key',
  `async_id` bigint NOT NULL DEFAULT '0' COMMENT 'Async Request ID',
  `error_data` longtext COMMENT 'Execution Error Information',
  `create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Creation Time',
  PRIMARY KEY (`id`) USING BTREE,
  KEY `idx_async_id` (`async_id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Async Log Table';

Async Strategy

Security Levels

Execution Status

Flowchart

Apollo Configuration

# Switch: disabled by default
async.enabled=true

# Application name
spring.application.name=xxx

# Data source (Druid)
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/fc_async?useUnicode=true&characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull&useSSL=false&allowMultiQueries=true&rewriteBatchedStatements=true
spring.datasource.username=user
spring.datasource.password=xxxx
spring.datasource.filters=config
spring.datasource.connectionProperties=config.decrypt=true;config.decrypt.key=yyy

# Static resources
spring.resources.add-mappings=true
spring.resources.static-locations=classpath:/static/

# Core thread count
async.executor.thread.corePoolSize=10
# Max thread count
async.executor.thread.maxPoolSize=50
# Queue capacity
async.executor.thread.queueCapacity=10000
# Keep‑alive seconds
async.executor.thread.keepAliveSeconds=600

# Delete record after success (default true)
async.exec.deleted=true

# Queue name prefix (default application name)
async.topic=${spring.application.name}_async_queue

# Retry count (default 5)
async.exec.count=5

# Retry query limit
async.retry.limit=100

# Compensation query limit
async.comp.limit=100

# Login interception (default false)
async.login=false

Usage

1. Async switch

scm.async.enabled=true

2. Annotate methods that need async execution (must be Spring‑proxied)

@AsyncExec(type = AsyncExecEnum.SAVE_ASYNC, remark = "Data Dictionary")

3. Manual handling address

http://localhost:8004/async/index.html

Notes

1. Application name

spring.application.name

2. Queue name

${async.topic:${spring.application.name}}_async_queue

3. Business logic must be idempotent

4. One application shares a single queue

Self‑produce, self‑consume.

5. Scheduled tasks

Async retry task (retry every 2 minutes, configurable retry count)

Async compensation task (compensate hourly for records older than one hour)

Effect Demonstration

Code Repository

https://github.com/xiongyanokok/fc-async