Migration to spring-boot 4+ and spring-batch 6+ ... concurrency effects with OptimisticLockingFailureException

[ahoehma]

Hi all,

I'm currently migrate my service from springboot 3.5.x to 4.0.0 and so also springbatch to 6+.

And I'm struggle little bit with the "new" setup :-)

One thing I can't get working is a random effect if I'm using my own BatchTaskExecutor.

My old configuration was like this:

@Configuration
@EnableConfigurationProperties(org.springframework.boot.autoconfigure.batch.BatchProperties.class)
@EnableBatchProcessing(
  dataSourceRef = "batchDataSource",
  taskExecutorRef = "batchTaskExecutor",
  transactionManagerRef = "batchTransactionManager")
public class BatchConfiguration {

  @Bean(destroyMethod = "shutdown")
  @BatchTaskExecutor
  TaskExecutor batchTaskExecutor(final UpdateProperties updateProperties) {
    final ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(updateProperties.getMaxUpdatesPerTime());
    executor.setMaxPoolSize(updateProperties.getMaxUpdatesPerTime());
    executor.setQueueCapacity(updateProperties.getTaskQueueCapacity());
    executor.setThreadNamePrefix("cc-update-job-");
    return executor;
  }

  @Bean
  PlatformTransactionManager batchTransactionManager(@Qualifier("batchDataSource") final DataSource dataSource) {
    return new JdbcTransactionManager(dataSource);
  }

  @Bean
  BatchDataSourceScriptDatabaseInitializer batchDataSourceInitializer(@Qualifier("batchDataSource") final DataSource dataSource, final BatchProperties properties) {
    return new BatchDataSourceScriptDatabaseInitializer(dataSource, properties.getJdbc());
  }

  ... then some Step with tasklet...

@Bean
  Step prepareUpdate(final JobRepository jobRepository, final PlatformTransactionManager batchTransactionManager, final PrepareService prepareService) {
    return new StepBuilder("prepare-update", jobRepository)
        .tasklet((stepContribution, chunkContext) -> {
          try {
            final String jobName = chunkContext.getStepContext().getStepExecution().getJobExecution().getJobInstance().getJobName();
            final ExecutionContext ec = chunkContext.getStepContext().getStepExecution().getJobExecution().getExecutionContext();
            final JobParameters jobParameters = chunkContext.getStepContext().getStepExecution().getJobParameters();
            final GenerateSettings settings = createGenerateSettings(ec, jobParameters);
            prepareService.prepareUpdate(settings, jobName, ec, jobParameters);
            return RepeatStatus.FINISHED;
          } catch (final Exception e) {
            throw new UnexpectedJobExecutionException("Can't prepare update", e);
          }
        }, batchTransactionManager)
        .build();
  }

Now I have this configuration. If I keep my own "batchTaskExecutor" then I have random "OptimisticLockingFailureException"
like this ...

org.springframework.dao.OptimisticLockingFailureException: Attempt to update job execution id=2 with wrong version (0), where current version is 1
	at org.springframework.batch.core.repository.dao.jdbc.JdbcJobExecutionDao.updateJobExecution(JdbcJobExecutionDao.java:301)
	at org.springframework.batch.core.repository.support.SimpleJobRepository.update(SimpleJobRepository.java:145)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:359)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:158)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:370)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:222)
	at jdk.proxy2/jdk.proxy2.$Proxy248.update(Unknown Source)
	at org.springframework.batch.core.job.AbstractJob.updateStatus(AbstractJob.java:420)
	at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:289)
	at org.springframework.batch.core.launch.support.TaskExecutorJobLauncher$1.run(TaskExecutorJobLauncher.java:220)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

I also need to remove "@EnableBatchProcessing" at all .. if not other effects happens .. i.e. database is not initialized ... because "BatchDataSourceScriptDatabaseInitializer " is also gone.

@Configuration
public class BatchConfiguration {

  // my own custom datasource is marked with @BatchDataSource (the same but cleaner then my old @Qualifier("batchDataSource"))
  @Bean
  @BatchTransactionManager
  PlatformTransactionManager batchTransactionManager(@BatchDataSource final DataSource dataSource) {
    return new JdbcTransactionManager(dataSource);
  }

  @Bean
  JobRegistry jobRegistry() {
    return new MapJobRegistry();
  }

  ... then some Step with tasklet...

@Bean
  Step prepareUpdate(final JobRepository jobRepository, final PlatformTransactionManager batchTransactionManager, final PrepareService prepareService) {
    return new StepBuilder("prepare-update", jobRepository)
        .tasklet((stepContribution, chunkContext) -> {
          try {
            final String jobName = chunkContext.getStepContext().getStepExecution().getJobExecution().getJobInstance().getJobName();
            final ExecutionContext ec = chunkContext.getStepContext().getStepExecution().getJobExecution().getExecutionContext();
            final JobParameters jobParameters = chunkContext.getStepContext().getStepExecution().getJobParameters();
            final GenerateSettings settings = createGenerateSettings(ec, jobParameters);
            prepareService.prepareUpdate(settings, jobName, ec, jobParameters);
            return RepeatStatus.FINISHED;
          } catch (final Exception e) {
            throw new UnexpectedJobExecutionException("Can't prepare update", e);
          }
        }, batchTransactionManager)
        .build();
  }
 
}

Where can I check for more docu about migration ... the 6+ version have a lot of deprecated classes etc. I can't find a clean example etc. May someone can guide me to a docu?

My main concern is that if I'm using the default "SyncTaskExecutor" is my rest-api (which starts my jobs) are still parallel?

Kind regards
Andreas

[fmbenhassine]

Hi Andreas,

Thank you for opening this discussion. I will help you with your migration to Spring Boot 4 / Spring Batch 6.

I also need to remove "@EnableBatchProcessing" at all

This was already the case in Spring Boot 3: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.0-Migration-Guide#enablebatchprocessing-is-now-discouraged

Are you upgrading from an earlier version of Spring Boot? I really recommend to first upgrade to the latest Spring Boot 3.x before migrating to v4 (ie one major at a time)

If I keep my own "batchTaskExecutor" then I have random "OptimisticLockingFailureException"

This might be an issue in v6 indeed: #5106. I planned to fix in the upcoming 6.0.2

Where can I check for more docu about migration ... the 6+ version have a lot of deprecated classes etc. I can't find a clean example etc. May someone can guide me to a docu?

• Spring Batch migration guide: https://github.com/spring-projects/spring-batch/wiki/Spring-Batch-6.0-Migration-Guide
• Spring Boot migration guide: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-4.0-Migration-Guide#spring-batch
• Getting started guide: https://spring.io/guides/gs/batch-processing

Best regards,
Mahmoud