Master Spring Batch with Spring Boot: Build Scalable Batch Jobs Step‑by‑Step
This guide walks through setting up Spring Batch 4.2.7 with Spring Boot 2.4.12, covering architecture, configuration, job, step, reader, processor, writer, listeners, and deployment, demonstrating how to read CSV data, process it, and persist results to a MySQL database.
Environment: Spring Boot 2.4.12 + Spring Batch 4.2.7
Spring Batch is a lightweight, fully Spring‑based batch processing framework suitable for enterprise‑level data processing. It builds on POJOs and the familiar Spring ecosystem, providing features such as logging, transaction management, job statistics, restart, skip, and resource management.
Typical business scenarios include:
Periodic batch submission.
Parallel batch processing.
Phase‑based, enterprise‑message‑driven processing.
Large‑scale parallel batch jobs.
Manual or scheduled restart after failures.
Ordered processing of related steps (extendable to workflow‑driven batches).
Partial processing with record skipping (e.g., on rollback).
Whole‑batch transactions for small batches or existing stored procedures/scripts.
Technical goals:
Allow batch developers to focus on business logic while the framework handles infrastructure.
Clear separation of concerns among infrastructure, batch execution environment, and batch applications.
Provide a common core execution service that can be implemented across projects.
Offer simple, out‑of‑the‑box core execution interfaces with default implementations.
Leverage the Spring framework at all layers for easy configuration, customization, and extension.
Ensure existing core services are replaceable or extensible without impacting the infrastructure layer.
Supply a straightforward deployment model using a Maven‑built JAR that is completely separated from the application.
Spring Batch architecture:
The layered architecture highlights three main components: the application layer (custom batch jobs and code), the core layer (runtime classes such as JobLauncher, Job, and Step), and the infrastructure layer (common readers, writers, and services like RetryTemplate). Both the application and core build on the shared infrastructure.
Development process example: read a CSV file, process the data, and store it in a MySQL database.
Dependency declaration
<code><dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
<version>6.0.7.Final</version>
</dependency></code>Application configuration (application.yml)
<code>spring:
datasource:
driverClassName: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://localhost:3306/batch?serverTimezone=GMT%2B8
username: root
password: ******
type: com.zaxxer.hikari.HikariDataSource
hikari:
minimumIdle: 10
maximumPoolSize: 200
autoCommit: true
idleTimeout: 30000
poolName: MasterDatabookHikariCP
maxLifetime: 1800000
connectionTimeout: 30000
connectionTestQuery: SELECT 1
---
spring:
jpa:
generateDdl: false
hibernate:
ddlAuto: update
openInView: true
show-sql: true
---
spring:
batch:
job:
enabled: false # whether to auto‑run jobs
initialize-schema: always # auto‑create DB scripts</code>Enable batch processing
<code>@Configuration
@EnableBatchProcessing
public class BatchConfig extends DefaultBatchConfigurer {
}</code>Job launcher configuration
<code>@Override
protected JobLauncher createJobLauncher() throws Exception {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(createJobRepository());
jobLauncher.afterPropertiesSet();
return jobLauncher;
}</code>Job repository configuration
<code>@Resource
private PlatformTransactionManager transactionManager;
@Override
protected JobRepository createJobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDatabaseType("mysql");
factory.setTransactionManager(transactionManager);
factory.setDataSource(dataSource);
factory.afterPropertiesSet();
return factory.getObject();
}</code>Define the Job
<code>@Bean
public Job myJob(JobBuilderFactory builder, @Qualifier("myStep") Step step) {
return builder.get("myJob")
.incrementer(new RunIdIncrementer())
.flow(step)
.end()
.listener(jobExecutionListener)
.build();
}</code>ItemReader (CSV file)
<code>@Bean
public ItemReader<Person> reader() {
FlatFileItemReader<Person> reader = new FlatFileItemReader<>();
reader.setResource(new ClassPathResource("cvs/persons.cvs"));
reader.setLineMapper(new DefaultLineMapper<Person>() {{
setLineTokenizer(new DelimitedLineTokenizer(",") {{
setNames("id", "name");
}});
setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{
setTargetType(Person.class);
}});
}});
return reader;
}</code>ItemProcessor
<code>@Bean
public ItemProcessor<Person, Person2> processorPerson() {
return new ItemProcessor<Person, Person2>() {
@Override
public Person2 process(Person item) throws Exception {
Person2 p = new Person2();
p.setId(item.getId());
p.setName(item.getName() + ", pk");
return p;
}
};
}</code>ItemWriter (JPA)
<code>@Resource
private Validator<Person> validator;
@Resource
private EntityManagerFactory entityManagerFactory;
@Bean
public ItemWriter<Person2> writerPerson() {
JpaItemWriterBuilder<Person2> builder = new JpaItemWriterBuilder<>();
builder.entityManagerFactory(entityManagerFactory);
return builder.build();
}</code>Step definition
<code>@Bean
public Step myStep(StepBuilderFactory stepBuilderFactory,
ItemReader<Person> reader,
ItemWriter<Person> writer,
ItemProcessor<Person, Person> processor) {
return stepBuilderFactory.get("myStep")
.<Person, Person>chunk(2)
.reader(reader)
.faultTolerant()
.retryLimit(3)
.retry(Exception.class)
.skip(Exception.class)
.skipLimit(2)
.listener(new MyReadListener())
.processor(processor)
.writer(writer)
.faultTolerant()
.skip(Exception.class)
.skipLimit(2)
.listener(new MyWriteListener())
.build();
}</code>Listeners
<code>public class MyReadListener implements ItemReadListener<Person> {
private Logger logger = LoggerFactory.getLogger(MyReadListener.class);
@Override public void beforeRead() {}
@Override public void afterRead(Person item) {
System.out.println("reader after: " + Thread.currentThread().getName());
}
@Override public void onReadError(Exception ex) {
logger.info("读取数据错误：{}", ex);
}
}
@Component
public class MyWriteListener implements ItemWriteListener<Person> {
private Logger logger = LoggerFactory.getLogger(MyWriteListener.class);
@Override public void beforeWrite(List<? extends Person> items) {}
@Override public void afterWrite(List<? extends Person> items) {
System.out.println("writer after: " + Thread.currentThread().getName());
}
@Override public void onWriteError(Exception exception, List<? extends Person> items) {
try {
logger.info(String.format("%s%n", exception.getMessage()));
for (Person item : items) {
logger.info(String.format("Failed writing BlogInfo : %s", item.toString()));
}
} catch (Exception e) {
e.printStackTrace();
}
}
}</code>Entity class
<code>@Entity
@Table(name = "t_person")
public class Person {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Integer id;
private String name;
}</code>Launch the job via REST controller
<code>@RestController
@RequestMapping("/demo")
public class DemoController {
@Resource
@Qualifier("myJob")
private Job job;
@Resource
private JobLauncher launcher;
@GetMapping("/index")
public Object index() {
JobParameters jobParameters = new JobParametersBuilder().toJobParameters();
try {
launcher.run(job, jobParameters);
} catch (JobExecutionAlreadyRunningException | JobRestartException |
JobInstanceAlreadyCompleteException | JobParametersInvalidException e) {
e.printStackTrace();
}
return "success";
}
}</code>The service starts, automatically creates the required table, runs the batch job, and populates the database. The following screenshots show the generated table and data.
All steps completed successfully.
Spring Full-Stack Practical Cases
Full-stack Java development with Vue 2/3 front-end suite; hands-on examples and source code analysis for Spring, Spring Boot 2/3, and Spring Cloud.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.