This documentation is also available as multiple HTML files and as PDF and EPUB documents.

Spring Batch Introduction

Many applications within the enterprise domain require bulk processing to perform business operations in mission-critical environments. These business operations include:

  • Automated, complex processing of large volumes of information that is most efficiently processed without user interaction. These operations typically include time-based events (such as month-end calculations, notices, or correspondence).

  • Periodic application of complex business rules processed repetitively across very large data sets (for example, insurance benefit determination or rate adjustments).

  • Integration of information that is received from internal and external systems that typically requires formatting, validation, and processing in a transactional manner into the system of record. Batch processing is used to process billions of transactions every day for enterprises.

Spring Batch is a lightweight, comprehensive batch framework designed to enable the development of robust batch applications that are vital for the daily operations of enterprise systems. Spring Batch builds upon the characteristics of the Spring Framework that people have come to expect (productivity, POJO-based development approach, and general ease of use), while making it easy for developers to access and use more advanced enterprise services when necessary. Spring Batch is not a scheduling framework. There are many good enterprise schedulers (such as Quartz, Tivoli, Control-M, and others) available in both the commercial and open source spaces. Spring Batch is intended to work in conjunction with a scheduler rather than replace a scheduler.

Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging and tracing, transaction management, job processing statistics, job restart, skip, and resource management. It also provides more advanced technical services and features that enable extremely high-volume and high performance batch jobs through optimization and partitioning techniques. You can use Spring Batch in both simple use cases (such as reading a file into a database or running a stored procedure) and complex, high volume use cases (such as moving high volumes of data between databases, transforming it, and so on). High-volume batch jobs can use the framework in a highly scalable manner to process significant volumes of information.

Background

While open source software projects and associated communities have focused greater attention on web-based and microservices-based architecture frameworks, there has been a notable lack of focus on reusable architecture frameworks to accommodate Java-based batch processing needs, despite continued needs to handle such processing within enterprise IT environments. The lack of a standard, reusable batch architecture has resulted in the proliferation of many one-off, in-house solutions developed within client enterprise IT functions.

SpringSource (now VMware) and Accenture collaborated to change this. Accenture’s hands-on industry and technical experience in implementing batch architectures, SpringSource’s depth of technical experience, and Spring’s proven programming model together made a natural and powerful partnership to create high-quality, market-relevant software aimed at filling an important gap in enterprise Java. Both companies worked with a number of clients who were solving similar problems by developing Spring-based batch architecture solutions. This input provided some useful additional detail and real-life constraints that helped to ensure the solution can be applied to the real-world problems posed by clients.

Accenture contributed previously proprietary batch processing architecture frameworks to the Spring Batch project, along with committer resources to drive support, enhancements, and the existing feature set. Accenture’s contribution was based upon decades of experience in building batch architectures with the last several generations of platforms: COBOL on mainframes, C++ on Unix, and, now, Java anywhere.

The collaborative effort between Accenture and SpringSource aimed to promote the standardization of software processing approaches, frameworks, and tools enterprise users can consistently use when creating batch applications. Companies and government agencies desiring to deliver standard, proven solutions to their enterprise IT environments can benefit from Spring Batch.

Usage Scenarios

A typical batch program generally:

  • Reads a large number of records from a database, file, or queue.

  • Processes the data in some fashion.

  • Writes back data in a modified form.

Spring Batch automates this basic batch iteration, providing the capability to process similar transactions as a set, typically in an offline environment without any user interaction. Batch jobs are part of most IT projects, and Spring Batch is the only open source framework that provides a robust, enterprise-scale solution.

Business Scenarios

Spring Batch supports the following business scenarios:

  • Commit batch process periodically.

  • Concurrent batch processing: parallel processing of a job.

  • Staged, enterprise message-driven processing.

  • Massively parallel batch processing.

  • Manual or scheduled restart after failure.

  • Sequential processing of dependent steps (with extensions to workflow-driven batches).

  • Partial processing: skip records (for example, on rollback).

  • Whole-batch transaction, for cases with a small batch size or existing stored procedures or scripts.

Technical Objectives

Spring Batch has the following technical objectives:

  • Let batch developers use the Spring programming model: Concentrate on business logic and let the framework take care of the infrastructure.

  • Provide clear separation of concerns between the infrastructure, the batch execution environment, and the batch application.

  • Provide common, core execution services as interfaces that all projects can implement.

  • Provide simple and default implementations of the core execution interfaces that can be used “out of the box”.

  • Make it easy to configure, customize, and extend services, by using the Spring framework in all layers.

  • All existing core services should be easy to replace or extend, without any impact to the infrastructure layer.

  • Provide a simple deployment model, with the architecture JARs completely separate from the application, built by using Maven.