Batch processing is about repetitive actions - either as a simple
optimisation, or as part of a job. To strategise and generalise the
repetition, and provide what amounts to an iterator framework, Spring
Batch has the RepeatOperations
interface. The
RepeatOperations
interface looks like this:
public interface RepeatOperations { ExitStatus iterate(RepeatCallback callback) throws RepeatException; }
where the callback is a simple interface that allows you to insert some business logic to be repeated
public interface RepeatCallback { ExitStatus doInIteration(RepeatContext context) throws Exception; }
The callback is executed repeatedly, until the
implementation decides that the iteration should end. The return value in
these interfaces is a special form of extendable enumeration (not a true
enumeration because users are free to create new values). An
ExitStatus
is immutable and conveys information to
the caller of the repeat operations about whether there is any more work
to do. Generally speaking, implementations of
RepeatOperations
should inspect the
ExitStatus
and use it as part of the decision to
end the iteration. Any callback that wishes to signal to the caller that
there is no more work to do can return
ExitStatus.FINISHED
.
The simplest general purpose implementation of
RepeatOperations
is
RepeatTemplate
. It could be used like this
RepeatTemplate template = new RepeatTemplate(); template.setCompletionPolicy(new FixedChunkSizeCompletionPolicy(2)); template.iterate(new RepeatCallback() { public ExitStatus doInIteration(RepeatContext context) { // Do stuff in batch... return ExitStatus.CONTINUABLE; } });
In the example we return ExitStatus.CONTINUABLE
to show
that there is more work to do. The callback can also return
ExitStatus.FINISHED
if it wants to signal to the caller that
there is no more work to do. Some iterations can be terminated by
considerations intrinsic to the work being done in the callback, others
are effectively infinite loops as far as the callback is concerned, and
the completion decision is delegated to an external policy as in the case
above.
The method parameter for the RepeatCallback
is a RepeatContext
. Many callbacks will simply
ignore the context, but if necessary it can be used as an attribute bag
to store transient data for the duration of the iteration. After the
iterate
method returns, the context will no
longer exist.
A RepeatContext
will have a parent context
if there is a nested iteration in progress. The parent context is
occasionally useful for storing data that need to be shared between
calls to iterate
. This is the case for instance
if you want to count the number of occurrences of an even in the
iteration and remember it across subsequent calls.
ExitStatus
is used by Spring Batch to
indicate whether processing has finished, and if so whether or not is
was successful. It is also used to carry textual information about the
end state of a batch or iteration, in the form of an exit code and a
description of the status in freeform text. These are the properties of
an ExitStatus
:
Table 5.1. ExitStatus properties
Property Name | Type | Description |
continuable | boolean | true if there is more work to do |
exitCode | String | Short code describing the exit status, e.g. CONTINUABLE, FINISHED, FAILED |
exitDescription | String | Long description of the exit status, could be a stack trace for example. |
ExitStatus
values are designed to be
flexible, so that they can be created with any code and description the
user needs. Spring Batch comes with some standard values out of the box,
to support common use cases, but users are free to create their own
values, as long as the semantics of teh continuable
property are honoured.
ExitStatus values can also be combined with valrious operators built into the class as methods. You can add an exit code, or description, or combine the continuable values with logical AND using methods in ExitStatus. You can also combine two ExitStatus values with the and method taking ExitStatus as a parameter. The effect of this is to do a logical AND on the continuable flag, concatenate the descriptions and replace the exit code with the new value, as long as the result is continuable, or the input is not continuable. This has the effect of maintaining the semantics of the continuable flag, but not making any "surprising" changes to the exit code (e.g. it never becomes CONTINUABLE when it was already FINISHED, unless someone does something wilful, like pass in a value that is not continuable, but with a code of CONTINUABLE).
Inside a RepeatTemplate
the termination of
the loop in the iterate
method is determined by a
CompletionPolicy
which is also a factory for the
ReapeatContext
. The
RepeatTemplate
has the reponsibility to use the
current policy to create a RepeatContext
and pass
that in to the RepeatCallback
at every stage in the
iteration. After a callback completes its
doInIteration
the
RepeatTemplate
has to make a call to the
CompletionPolicy
to ask it to update its state
(which will be stored in the RepeatContext
), then
it asks the policy if the iteration is complete.
Spring Batch provides some simple general purpose implementations of
CompletionPolicy
, for example the
SimpleCompletionPolicy
used in the example above.
The SimpleCompletionPolicy
just allows an execution
up to a fixed number of times (with ExitStatus.FINISHED
forcing early completion at any time).
Users might need to implement their own completion policies for more complicated decisions, e.g. a batch processing window that prevents batch jobs from executing once the online systems are in use.
If there is an exception thrown inside a
RepeatCallback
, the
RepeatTemplate
consults an
ExceptionHandler
which can decide whether or not to
re-throw the exception.
public interface ExceptionHandler { void handleException(RepeatContext context, Throwable throwable) throws RuntimeException; }
A common use case is to count the number of exceptions of a
given type, and fail when a limit is reached. For this purpose Spring
Batch provides the SimpleLimitExceptionHandler
and
slightly more flexible
RethrowOnThresholdExceptionHandler
. The
SimpleLimitExceptionHandler
has a limit property
and an exception type that should be compared with the current exception -
all subclasses of the provided type are also counted. Exceptions of the
given type are ignored until the limit is reached, and then rethrown.
Those of other types are always rethrown.
An important optional property of the
SimpleLimitExceptionHandler
is the boolean flag
useParent
. It is false by default, so the limit is only
accounted for in the current RepeatContext
. When
set to true, the limit is kept across sibling contexts in a nested
iteration (e.g. a set of chunks inside a step).
Often it is useful to be able to receive additional callbacks for
cross cutting concerns across a number of different iterations. For this
purpose Spring Batch provides the RepeatListener
interface. The RepeatTemplate
allows users to
register RepeatListener
s, and they will be given
callbacks with the RepeatContext
and
ExitStatus
where available during the
iteration.
The interface looks like this:
public interface RepeatListener { void before(RepeatContext context); void after(RepeatContext context, ExitStatus result); void open(RepeatContext context); void onError(RepeatContext context, Throwable e); void close(RepeatContext context); }
The open
and
close
callbacks come before and after the entire
iteration, and before
,
after
and onError
apply
to the individual RepeatCallback calls.
Note that when there is more than one listener, they are in a list,
so there is an order. In this case open
and
before
are called in the same order, and
after
, onError
and
close
are called in reverse order.
Implementations of RepeatOperations
are not
restricted to executing the callback sequentially. It is quite important
that some implementations are able to execute their callbacks in parallel.
To this end Spring Batch provides the
TaskExecutorRepeatTemplate
, which uses the Spring
TaskExecutor
strategy to run the
RepeatCallback
. The default is to use a
SynchronousTaskExecutor, which has the effect of executing the whole
iteration in the same thread (the same as a normal
RepeatTemplate
).
Sometimes there is some business processing that you know you want
to repeat every time it happens. The classic example of this is the
optimization of a message pipeline - it is more efficient to process a
batch of messages, if they are arriving frequently, than to bear the cost
of a separate transaction for every message. Spring Batch provides an AOP
interceptor that wraps a method call in a
RepeatOperations
for just this purpose. The
RepeatOperationsInterceptor
executes the
intercepted method and repeats according to the
CompetionPolicy
in the provided
RepeatTemplate
.
Here is an example of declarative iteration using the Spring AOP
namespace to repeat a service call to a method called
processMessage
(for more detail on how to
configure AOP interceptors see the Spring User Guide):
<aop:config> <aop:pointcut id="transactional" expression="execution(* com...*Service.processMessage(..))" /> <aop:advisor pointcut-ref="transactional" advice-ref="retryAdvice" order="-1"/> </aop:config> <bean id="retryAdvice" class="org.springframework.batch.repeat.interceptor.RepeatOperationsInterceptor"/>
The example above uses a default
RepeatTemplate
inside the interceptor. To change
the policies, listeners etc. you only need to inject an instance of
RepeatTemplate
into the interceptor.
If the intercepted method returns void
then the
interceptor always returns ExitStatus.CONTINUABLE (so there is a danger of
an infinite loop if the CompletionPolicy
does not
have a finite end point). Otherwise it returns
ExitStatus.CONTINUABLE
until the return value from the
intercepted method is null, at which point it returns
ExitStatus.FINISHED
. So the business logic inside the target
method can signal that there is no more work to do by returning
null
, or by throwing an exception that is re-thrown by the
ExceptionHandler
in the provided
RepeatTemplate
.