Main Use Cases

The easiest way to get a quick overview of Spring Batch Admin is to see some screenshots of the main use cases.The user interface is a web application (built with Spring MVC).

Inspect Jobs

The user can inspect the jobs that are known to the system. Jobs are either launchable or non-launchable (in the screenshot they are all launchable). The distinction is that a launchable job is defined and configured in the application itself, whereas a non-launchable job is detected as state left by the execution of a job in another process. (Spring Batch uses a relational database to track the state of jobs and steps, so historic executions can be queried to show the non-launchable jobs.)

Launch Job

Launchable jobs can be launched from the user interface with job parameters provided as name value pairs, or by an incrementer configured into the application.

Inspect Executions

Once a job is executing, or has executed, this view can be used to see the most recent executions, and a brief summary of their status (STARTED, COMPLETED, FAILED, etc.).

Each individual execution has a more detailed view (shown above), and from there the user can click down to a view of each of the step executions in the job (only one in this case). A common reason for wanting to do this is to see the cause of a failure.

The top of the step execution detail view shows the history of the execution of this step across all job executions. This is useful for getting a statistical feel for performance characteristics. A developer running a job in an integration test environment might use the statistics here to compare different parameterisations of a job, to see what effect is of changing (for instance) the commit interval in an item processing step.

The bottom of the step execution view has the detailed meta-data for the step (status, read count, write count, commit count, etc.) as well as an extract of the stack trace from any exception that caused a failure of the step (as in the example shown above).

Stop an Execution

A job that is executing can be stopped by the user (whether or not it is launchable). The stop signal is sent via the database and once detected by Spring Batch in whatever process is running the job, the job is stopped (status moves from STOPPING to STOPPED) and no further processing takes place.

Installing Spring Batch Admin

There is a sample project in the distribution - it's a regular war file project which you can build with Maven (mvn install and look in the target directory). The sample can also be created with a couple of clicks in the SpringSource Tool Suite (STS): go to File->New->Spring Template Project, and select the Spring Batch Admin Webapp from the list (if it isn't there upgrade STS). In STS you can right click on the project and Run As->Run On Server choose Tomcat or tc Server, and the app will deploy and open a web browser (e.g. localhost:8080/spring-batch-admin-sample). Check out the features for launching jobs and inspecting job executions. If the STS project looks out of date (look at the version numbers in the pom.xml) then you might be able to upgrade by changing the POM, or else you may have to install a nightly update of STS or wait for the next release.

Running Spring Batch Admin

If you are deploying Spring Batch Admin as a WAR file (either the sample or one you rolled yourself), just deploy it as normal to a servlet container like Pivotal tcServer.

If the WAR file is called spring-batch-admin-sample.war (it may have a version number appended), then when you load the application in a browser at http://localhost:8080/spring-batch-admin-sample)) it should show a listing of the contents of the application:

This page is a list of URLs, or URL patterns, along with the HTTP method (GET, POST, etc.) that you need to use to carry out an operation on that resource. If the URL is a pattern it has braces ({}) in it to represent the value of a parameter that needs to be supplied by the user. Users of Spring REST support in MVC will recognise this convention, and also the idea that a GET represents a request for information, and a POST is an "insert" operation. So, for example, to request a list of all running Job executions you must GET the resource at /jobs/executions (relative to the batch/ servlet in the application). And to launch a new job execution you POST to /jobs/jobName (with "{jobName}" replaced with the name of a launchable job).

Links on the home page that are not parameterised and require a GET method are clickable in the browser and take you to that page in the application. The others are all navigable from there.

Menus

The basic menus for the application out of the box are

• Home - the URL list or site map listing the interface for the application
• Jobs - list and launch jobs. Only jobs which are either configured in the application or have been executed in other processes against the same database show up here.
• Executions - view and interact with running Job and Step executions.
  • Files - upload and list input files and configuration files for jobs.

You can learn how to extend the content of the application including the menus in a later section of this guide.

JSON API

Spring Batch Admin can act as a JSON web service. All the main HTML UI features can be accessed by JSON clients. In general, the recipe for doing this is to take a normal HTML URL from the UI application and add a .json suffix. All the JSON endpoints are listed on the home page of the sample application.

The examples below all use curl from the command line, but the same resources could be used with an Ajax request from a browser, or with Spring's RestTemplate (see the integration tests in the project source code for examples of the latter).

$ curl -s http://localhost:8080/spring-batch-admin-sample/batch/jobs.json
{"jobs" : { 
    "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs.json",
    "registrations" : {
        "infinite" : {
            "name" : "infinite",
            "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs/infinite.json",
            "description" : "No description",
            "executionCount" : 0,
            "launchable" : true,
            "incrementable" : false
        },
        "job1" : {
            "name" : "job1",
                        ...
        },
        "job2" : {
            "name" : "job2",
                        ...
        }
     }
  }
}

$ curl -s http://localhost:8080/sprin-batch-admin-sample/batch/jobs/job1.json
{"job" : { 
    "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs/job1.json",
    "name" : "job1",
    "jobInstances" : {
        "0" : {
            "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs/job1/0/executions.json",
            "executionCount" : 1,
            "lastJobExecution" : "/spring-batch-admin-sample/batch/jobs/executions/0.json",
            "lastJobExecutionStatus" : "FAILED",
            "jobParameters" : {
               "run.count(long)" : "0"
            }
        }
     }
  }
}

$ curl -d jobParameters=fail=false http://localhost:8080/spring-batch-admin-sample/batch/jobs/job1.json
{"jobExecution" : { 
    "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs/executions/2.json",
    "id" : "2",
    "status" : "STARTING",
    "startTime" : "",
    "duration" : "",
    "exitCode" : "UNKNOWN",
    "exitDescription" : "",
    "jobInstance" : { "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs/job1/1.json" },
    "stepExecutions" : {
    }
  }
}

$ curl -s http://localhost:8080/spring-batch-admin-sample/batch/jobs/executions/2.json
{"jobExecution" : { 
    "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs/executions/2.json",
    "id" : "3",
    "status" : "COMPLETED",
    "startTime" : "17:44:52",
    "duration" : "00:00:00",
    "exitCode" : "COMPLETED",
    "exitDescription" : "",
    "jobInstance" : { "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs/job1/2.json" },
    "stepExecutions" : {
        "step1" : {
            "status" : "COMPLETED",
            "exitCode" : "COMPLETED",
            "id" : "3",
                "resource" : "http://localhost:8080/spring-batch-admin-sample/batch/jobs/executions/2/steps/2.json",
            "readCount" : "5",
            "writeCount" : "5",
            "commitCount" : "6",
            "rollbackCount" : "0",
            "duration" : "00:00:00"
            }
    }
  }
}

Configuration Upload

An interesting problem that has no universal good solution is: how do I change the configuration of a running Spring application? Spring Batch Admin has a configuration upload feature that solves this problem in a particular way.

Screenshots of the Basic Use Case

We start with a look at the Jobs view in the application. It shows a list of jobs that are either launchable or monitorable by the web application.

Now the plan is to upload a new Job configuration and see this view change. So we start with the "Files" menu in the top navigation, and then click on "Configuration". This presents a simple form to upload a file. If we point to a Spring XML configuration file, the view looks like this:

We press the "Upload" button and the configuration is uploaded and parsed and the Job instances in the uploaded file are registered for launching:

You can see a new entry in the job registry ("test-job") which is launchable in-process because the application has a reference to the Job. (Jobs which are not launchable were executed out of process, but used the same database for its JobRepository, so they show up with their executions in the UI.)

Monitoring and Management with JMX

Spring Batch Admin provides some nice features for monitoring and managing Batch jobs remotely using JMX. In addition the standard configuration has JMX MBeans enabled for the Spring Integration components within the application context.

Exposing Metrics for Job and Step Executions

All the basic JMX features are exposed in one place in the Manager jar, in META-INF/spring/batch/bootstrap/manager/jmx-context.xml. The Job and Step metrics come from one bean, which in turn registers a set of MBeans for the Job and Step metrics:

<bean id="batchMBeanExporter" class="org.springframework.batch.admin.jmx.BatchMBeanExporter">
        <property name="server" ref="mbeanServer" />
        <property name="jobService">
                <bean class="org.springframework.aop.framework.ProxyFactoryBean">
                        <property name="target" ref="jobService" />
                </bean>
        </property>
        <property name="defaultDomain" value="spring.application" />
</bean>

If you start the sample application with the usual command line options to expose the MBeanServer, e.g.

-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl.need.client.auth=false

then launch one of the jobs, and look at the MBeans in JConsole or your favourite JMX client, you will see the BatchMBeanExporter. As soon as you inspect its attributes, you will also see Job and Step Executions as well (they are lazily registered only when the basic attributes are first inspected):

The BatchMBeanExporter needs a reference to the JobService because it uses the JobRepository to extract most of its metrics. It also has a basic cache for the metric data, to prevent it having to go back to the repository every time it is polled. The default cache settings have an expiry timeout of 60 seconds (so new data will show up in JMX every 60 seconds). The cache can be re-configured by overriding the bean definition for the cacheInterceptor bean.

Job Execution Metrics

Once a Job has executed at least once and the BatchMBeanExporter attributes inspected (so the Job execution leaves an imprint on the JobRepository) its metrics will be available to JMX clients. The MBean for these metrics has an ObjectName in the form

spring.application:type=JobExectution,name=job1

where job1 is the job name.

Metrics include information about the latest execution (status, duration, exit description, etc.) and also some basic summary information (number of executions, number of failures etc.).

Step Execution Metrics

Once a step has executed at least once (and left an imprint on the JobRepository) its metrics will be available to JMX clients. The MBean for these metrics has an ObjectName in the form

spring.application:type=JobExectution,name=job1,step=step1

where job1 is the parent job name, and step1>> is the step name. Because of the form of the <<<ObjectName, most JMX clients present step execution metrics as a child of the job they belong to, making them easy to locate.

Metrics include information about the latest execution (status, duration, commit count, read and write counts, etc.) and also some basic summary information (number of executions, number of failures etc.).

Step Execution Timeouts and Monitoring

The metrics above can be used by JMX clients to monitor the performance of an application and send alerts if service levels are not being achieved. Many JMX clients can do this natively with no additional configuration in the Batch application.

In addition, a convenient JMX MonitorMBean is provided in Spring Batch Admin to allow users to receive notifications if a Step execution is overunning. To use this just add a bean definition to your Job configuration context, and refer to the Step you want to monitor e.g.

<bean id="j1.step1.monitor" class="org.springframework.batch.admin.jmx.StepExecutionServiceLevelMonitor">
        <property name="jobName" value="job1"/>
        <property name="stepName" value="step1"/>
        <property name="upperThreshold" value="30000"/>
        <property name="defaultDomain" value="spring.application"/>
</bean>

This will create an MBean with ObjectName in the form:

spring.application:type=GaugeMonitor,name=job1.step1.monitor

which can be exposed through the MBeanServer using the standard Spring context shortcut:

<context:mbean-export default-domain="spring.application" server="mbeanServer"/>        

When a step is taking too long to execute, a JMX notification is sent automatically by the MBeanServer to any clients that have registered for notifications. The notification comes with an embedded event in the form:

javax.management.monitor.MonitorNotification
  [source=spring.application:name=job1.step1.monitor,type=GaugeMonitor]
  [type=jmx.monitor.gauge.high]
  [message=]

where job1.step1.monitor is the name of the Spring bean definition used to define the monitor.

Environment Settings

The most likely thing you will want to customize is the location of the database. Spring Batch Admin ships with an embedded HSQLDB database, which is initialized on start up.

• To change the database type add a file to the application classpath called batch-[type].properties, where [type] is the database type you want to use (e.g. mysql, oracle, db2). Copy the contents of the batch-hsql.properties from the Manager jar and change the values to suit your environment. Then launch the application with a system property -DENVIRONMENT=[type].
• To stop the database from being wiped and re-created on start up just set batch.data.source.init=false (in the properties file or as a System property).

N.B. The use of the environment variable to switch on the database type is only a suggestion. You can change that and other things by overriding and adding configuration fragments to the Spring application context. See below for details.

Overriding Components from Spring Batch Admin

The system tries to provide some useful defaults for things like transaction manager, job repository, job registry etc. Most of these live in the manager jar in a special place: META-INF/spring/batch/bootstrap. If you want to override them, just add your own versions of the same bean definitions to a Spring XML config file in META-INF/spring/batch/override (these are guaranteed to load after the bootstrap files, so they can override default definitions). You could use this to override the data source definition as an alternative to the environment settings described above.

Add your Own Job Executions

To add job executions from another process just execute a job (e.g. from command line) against the same database that is used by Spring Batch Admin. The UI picks up the meta data from the usual Spring Batch tables.

<beans ...>

  <import resource="classpath*:/META-INF/spring/batch/bootstrap/manager/*.xml" />
  <import resource="classpath*:/META-INF/spring/batch/jobs/*.xml" />

  <!-- ensure job runs in foreground thread -->
  <bean id="jobLauncherTaskExecutor" class="org.springframework.core.task.SyncTaskExecutor"/>
  <!-- prevent loading of other jobs by overriding the loader in the main bootstrap context -->
  <bean id="jobLoader" class="java.lang.String"/>

</beans>

$ java org.springframework.batch.core.launch.support.CommandLineJobRunner \
  -next config/launch-context.xml jobName

Add your Own Jobs For Launching in the UI

Include Spring XML files in META-INF/spring/batch/jobs. Each file should be self-contained (except for Batch execution components like the jobRepository which are provided centrally), and carry one or more bean definitions for a Spring Batch Job. When the application starts these files are scanned and loaded as child contexts, and then the jobs are registered with a JobRegistry in the parent. Because they are child contexts you don't have to worry about name clashes between different XML files or different contributing JARs (except for the jobs which must have unique names).

As a convenience, the child contexts inherit property placeholders and AOP configuration from the parent (this is not the default behaviour for a child context in Spring). This means you can control those things centrally if you need to. Of course, the child can always create its own placeholder definition and AOP configuration, but these will not affect the parent or any of its siblings.

Design and Architecture

Spring Batch Admin is a layered, extensible application. Its main artifacts are 2 JAR files (libraries) containing all the content and business logic for the web application. To deploy them you need a deployment platform, like a WAR file in a servlet container. There is a sample application spring-batch-admin-sample which show how this works in practice.

Application Context Structure

There is a root or parent application context, and a child context for the UI components. These are both loaded in the standard Spring MVC way following directives in web.xml. There are also possibly several additional child contexts (of the main root) containing job configurations.

• The root context is loaded from a file in the manager JAR, but all that does is import from well known locations on the classpath:

  <import resource="classpath*:/META-INF/spring/batch/bootstrap/**/*.xml"/>
  <import resource="classpath*:/META-INF/spring/batch/override/**/*.xml"/>

  Note that the "override" location has no files in it in Spring Batch Admin distribution. This is a placeholder for users to add their own content.

• The UI child context is loaded in the same way from

  <import resource="classpath*:/META-INF/spring/batch/servlet/**/*.xml"/>

  N.B. the child context by default only loads when you visit the first page in the application. You should be able to see which files it actually loads from the log statements that come out on the server console.

• The job contexts are loaded by a special component (ClasspathXmlJobLoader from Spring Batch) in the parent context. It looks for individual files in the pattern classpath*/META-INF/spring/batch/jobs/*.xml and loads each file individually as its own self-contained context, registering any instance of Job it finds in the JobRegistry at the top level.

The job contexts are loaded in this way so that you can provide your own jobs in multiple (possibly many) JAR files included in the application, but you do not have to worry too much about duplicate bean definitions because only the Job names have to be unique. A job context inherits the AOP and PropertyPlaceholderConfigurer settings from the root context (not the standard behaviour of a child context) as a convenience, but you can add your own settings to apply locally as well.

Extending the UI

There are three extension points for the UI: changing or translating text, adding menus, and modifying the content of the existing pages.

<bean id="messageSource" class="org.springframework.context.support.ResourceBundleMessageSource">
   <property name="basename" value="messages"/>
</bean>

<bean name="jobs" parent="standard">
        <property name="attributes">
                <props merge="true">
                        <prop key="body">/manager/jobs/jobs.ftl</prop>
                        <prop key="titleCode">jobs.title</prop>
                        <prop key="titleText">Spring Batch Admin: Jobs</prop>
                </props>
        </property>
</bean>

<bean name="jobs" parent="standard">
   <property name="attributes">
      <props merge="true">
         <prop key="body">/manager/jobs/html/jobs.ftl</prop>
         <prop key="titleCode">jobs.title</prop>
         <prop key="titleText">Spring Batch Admin: Jobs</prop>
      </props>
   </property>
</bean>

<bean name="jobs.json" parent="standard.json">
   <property name="attributes">
      <props merge="true">
         <prop key="body">/manager/jobs/json/jobs.ftl</prop>
      </props>
   </property>
</bean>