Skip to main content
Code Review

Return to Question

added 832 characters in body
Source Link
Vasanth
Vasanth
  • 173
  • 7

Is1.Is there any approach to enhance the performance for processing 17,000 records, which currently takes approximately 21 minutes?

  1. Is it possible to utilize the parallelStream method within a Spring Batch item writer?

import org.springframework.batch.item.ItemWriter; public class EmployeeItemWritter implements ItemWriter{

    {
        @Override
        public void write(Chunk<? extends Employee> chunk) throws Exception {
                
                 long startTime = System.currentTimeMillis();
                 employeeList.parallelStream().forEach(employee -> { 
                 Map<String, Object> resultMap =employeeService.getEmpDetails(employee.getEmpId(),employee.getEmpTxnId);             
                 }); 
                 long endTime = System.currentTimeMillis();
                 System.out.println("Processing time in milleseconds "+(endTime-startTime));
        }        
                 
    }

Is there any approach to enhance the performance for processing 17,000 records, which currently takes approximately 21 minutes?

1.Is there any approach to enhance the performance for processing 17,000 records, which currently takes approximately 21 minutes?

  1. Is it possible to utilize the parallelStream method within a Spring Batch item writer?

import org.springframework.batch.item.ItemWriter; public class EmployeeItemWritter implements ItemWriter{

    {
        @Override
        public void write(Chunk<? extends Employee> chunk) throws Exception {
                
                 long startTime = System.currentTimeMillis();
                 employeeList.parallelStream().forEach(employee -> { 
                 Map<String, Object> resultMap =employeeService.getEmpDetails(employee.getEmpId(),employee.getEmpTxnId);             
                 }); 
                 long endTime = System.currentTimeMillis();
                 System.out.println("Processing time in milleseconds "+(endTime-startTime));
        }        
                 
    }
Source Link
Vasanth
Vasanth
  • 173
  • 7

Spring Batch Item Writer Calling Stored Procedure

While migrating a Spring application to Spring Batch, I encountered a scenario where I need to process chunks of records from the database and invoke a stored procedure for each chunk. However, each stored procedure call is taking approximately 500 milliseconds, and the performance improvement isn't significant.

Note : I should exclusively use stored procedures because the process involves handling records from over 30 tables

@Bean("simpletaskExecutor")
    public TaskExecutor simpletaskExecutor() {
        
         SimpleAsyncTaskExecutor asyncTaskExecutor = new  SimpleAsyncTaskExecutor("Async");
         asyncTaskExecutor.setConcurrencyLimit(concurrencyCount);
         asyncTaskExecutor.setThreadNamePrefix("can-batch-thread-"); 
         return asyncTaskExecutor;
        
    }


public class EmployeeItemWritter implements ItemWriter<Employee>{

  @Autowired
  private EmployeeService employeeService;
    
  @Override
    public void write(Chunk<? extends Employee> chunk) throws Exception {
    
    List<Employee> employeeList=(List<Employee>) chunk.getItems();
    
     for(Employee employee:employeeList) 
          {
              //long start = System.currentTimeMillis();
              callStoredProcedure(employee);
             // callStoredProcedureWithoutJPA(indvInvoiceConsCtl);
              //long end = System.currentTimeMillis();
              //System.out.println("Processing time in milleseconds "+(end-start));
          
          }
      
    
    }
    
    
    public void callStoredProcedure(Employee employee) {
        
        Map<String, Object> resultMap = employeeService.getEmployeeDetails(employee.getEmpId(),employee.getEmpTxnId());
        
        if(CollectionUtils.isEmpty(resultMap)) {
            log.error(ERROR_WHILE_EXECUTING_EMPLOYEE_DETAILS_PROCEDURE);
        }
    }

}

Is there any approach to enhance the performance for processing 17,000 records, which currently takes approximately 21 minutes?