Chapter 5. Repeat

5.1. RepeatTemplate

Batch processing is about repetitive actions - either as a simple optimisation, or as part of a job. To strategise and generalise the repetition, and provide what amounts to an iterator framework, Spring Batch has the RepeatOperations interface. The RepeatOperations interface looks like this:

public interface RepeatOperations {

    ExitStatus iterate(RepeatCallback callback) throws RepeatException;

}

where the callback is a simple interface that allows you to insert some business logic to be repeated

public interface RepeatCallback {

    ExitStatus doInIteration(RepeatContext context) throws Exception;

}

The callback is executed repeatedly, until the implementation decides that the iteration should end. The return value in these interfaces is a special form of extendable enumeration (not a true enumeration because users are free to create new values). An ExitStatus is immutable and conveys information to the caller of the repeat operations about whether there is any more work to do. Generally speaking, implementations of RepeatOperations should inspect the ExitStatus and use it as part of the decision to end the iteration. Any callback that wishes to signal to the caller that there is no more work to do can return ExitStatus.FINISHED.

The simplest general purpose implementation of RepeatOperations is RepeatTemplate. It could be used like this

RepeatTemplate template = new RepeatTemplate();

template.setCompletionPolicy(new FixedChunkSizeCompletionPolicy(2));

template.iterate(new RepeatCallback() {

    public ExitStatus doInIteration(RepeatContext context) {
        // Do stuff in batch...
        return ExitStatus.CONTINUABLE;
    }

});

In the example we return ExitStatus.CONTINUABLE to show that there is more work to do. The callback can also return ExitStatus.FINISHED if it wants to signal to the caller that there is no more work to do. Some iterations can be terminated by considerations intrinsic to the work being done in the callback, others are effectively infinite loops as far as the callback is concerned, and the completion decision is delegated to an external policy as in the case above.

5.1.1. RepeatContext

The method parameter for the RepeatCallback is a RepeatContext. Many callbacks will simply ignore the context, but if necessary it can be used as an attribute bag to store transient data for the duration of the iteration. After the iterate method returns, the context will no longer exist.

A RepeatContext will have a parent context if there is a nested iteration in progress. The parent context is occasionally useful for storing data that need to be shared between calls to iterate. This is the case for instance if you want to count the number of occurrences of an even in the iteration and remember it across subsequent calls.

5.1.2. ExitStatus

ExitStatus is used by Spring Batch to indicate whether processing has finished, and if so whether or not is was successful. It is also used to carry textual information about the end state of a batch or iteration, in the form of an exit code and a description of the status in freeform text. These are the properties of an ExitStatus:

Table 5.1. ExitStatus properties

Property NameTypeDescription
continuablebooleantrue if there is more work to do
exitCodeStringShort code describing the exit status, e.g. CONTINUABLE, FINISHED, FAILED
exitDescriptionStringLong description of the exit status, could be a stack trace for example.

ExitStatus values are designed to be flexible, so that they can be created with any code and description the user needs. Spring Batch comes with some standard values out of the box, to support common use cases, but users are free to create their own values, as long as the semantics of teh continuable property are honoured.

ExitStatus values can also be combined with valrious operators built into the class as methods. You can add an exit code, or description, or combine the continuable values with logical AND using methods in ExitStatus. You can also combine two ExitStatus values with the and method taking ExitStatus as a parameter. The effect of this is to do a logical AND on the continuable flag, concatenate the descriptions and replace the exit code with the new value, as long as the result is continuable, or the input is not continuable. This has the effect of maintaining the semantics of the continuable flag, but not making any "surprising" changes to the exit code (e.g. it never becomes CONTINUABLE when it was already FINISHED, unless someone does something wilful, like pass in a value that is not continuable, but with a code of CONTINUABLE).

5.2. Completion Policies

Inside a RepeatTemplate the termination of the loop in the iterate method is determined by a CompletionPolicy which is also a factory for the ReapeatContext. The RepeatTemplate has the reponsibility to use the current policy to create a RepeatContext and pass that in to the RepeatCallback at every stage in the iteration. After a callback completes its doInIteration the RepeatTemplate has to make a call to the CompletionPolicy to ask it to update its state (which will be stored in the RepeatContext), then it asks the policy if the iteration is complete.

Spring Batch provides some simple general purpose implementations of CompletionPolicy, for example the SimpleCompletionPolicy used in the example above. The SimpleCompletionPolicy just allows an execution up to a fixed number of times (with ExitStatus.FINISHED forcing early completion at any time).

Users might need to implement their own completion policies for more complicated decisions, e.g. a batch processing window that prevents batch jobs from executing once the online systems are in use.

5.3. Exception Handling

If there is an exception thrown inside a RepeatCallback, the RepeatTemplate consults an ExceptionHandler which can decide whether or not to re-throw the exception.

public interface ExceptionHandler {

    void handleException(RepeatContext context, Throwable throwable)
        throws RuntimeException;

}

A common use case is to count the number of exceptions of a given type, and fail when a limit is reached. For this purpose Spring Batch provides the SimpleLimitExceptionHandler and slightly more flexible RethrowOnThresholdExceptionHandler. The SimpleLimitExceptionHandler has a limit property and an exception type that should be compared with the current exception - all subclasses of the provided type are also counted. Exceptions of the given type are ignored until the limit is reached, and then rethrown. Those of other types are always rethrown.

An important optional property of the SimpleLimitExceptionHandler is the boolean flag useParent. It is false by default, so the limit is only accounted for in the current RepeatContext. When set to true, the limit is kept across sibling contexts in a nested iteration (e.g. a set of chunks inside a step).

5.4. Listeners

Often it is useful to be able to receive additional callbacks for cross cutting concerns across a number of different iterations. For this purpose Spring Batch provides the RepeatListener interface. The RepeatTemplate allows users to register RepeatListeners, and they will be given callbacks with the RepeatContext and ExitStatus where available during the iteration.

The interface looks like this:

public interface RepeatListener {
    void before(RepeatContext context);

    void after(RepeatContext context, ExitStatus result);

    void open(RepeatContext context);

    void onError(RepeatContext context, Throwable e);

    void close(RepeatContext context);
}

The open and close callbacks come before and after the entire iteration, and before, after and onError apply to the individual RepeatCallback calls.

Note that when there is more than one listener, they are in a list, so there is an order. In this case open and before are called in the same order, and after, onError and close are called in reverse order.

5.5. Parallel Processing

Implementations of RepeatOperations are not restricted to executing the callback sequentially. It is quite important that some implementations are able to execute their callbacks in parallel. To this end Spring Batch provides the TaskExecutorRepeatTemplate, which uses the Spring TaskExecutor strategy to run the RepeatCallback. The default is to use a SynchronousTaskExecutor, which has the effect of executing the whole iteration in the same thread (the same as a normal RepeatTemplate).

5.6. Declarative Iteration

Sometimes there is some business processing that you know you want to repeat every time it happens. The classic example of this is the optimization of a message pipeline - it is more efficient to process a batch of messages, if they are arriving frequently, than to bear the cost of a separate transaction for every message. Spring Batch provides an AOP interceptor that wraps a method call in a RepeatOperations for just this purpose. The RepeatOperationsInterceptor executes the intercepted method and repeats according to the CompetionPolicy in the provided RepeatTemplate.

Here is an example of declarative iteration using the Spring AOP namespace to repeat a service call to a method called processMessage (for more detail on how to configure AOP interceptors see the Spring User Guide):

<aop:config>
    <aop:pointcut id="transactional"
        expression="execution(* com...*Service.processMessage(..))" />
    <aop:advisor pointcut-ref="transactional"
        advice-ref="retryAdvice" order="-1"/>
</aop:config>

<bean id="retryAdvice"
    class="org.springframework.batch.repeat.interceptor.RepeatOperationsInterceptor"/>

The example above uses a default RepeatTemplate inside the interceptor. To change the policies, listeners etc. you only need to inject an instance of RepeatTemplate into the interceptor.

If the intercepted method returns void then the interceptor always returns ExitStatus.CONTINUABLE (so there is a danger of an infinite loop if the CompletionPolicy does not have a finite end point). Otherwise it returns ExitStatus.CONTINUABLE until the return value from the intercepted method is null, at which point it returns ExitStatus.FINISHED. So the business logic inside the target method can signal that there is no more work to do by returning null, or by throwing an exception that is re-thrown by the ExceptionHandler in the provided RepeatTemplate.