Spring Cloud Task Reference Guide

Spring Cloud Task uses a relational database to store the results of an executed task. While you can begin developing a task without a database (the status of the task is logged as part of the task repository’s updates), for production environments, you want to use a supported database. Spring Cloud Task currently supports the following databases:

Now we can create and test an application that prints Hello, World! to the console.

To do so:

To finish our application, we need to update the generated HelloworldApplication with the following contents so that it launches a Task.

While it may seem small, quite a bit is going on. For more about Spring Boot specifics, see the Spring Boot reference documentation.

Now we can open the application.properties file in src/main/resources. We need to configure two properties in application.properties:

The following example shows how to do both:

At this point, our application should work. Since this application is Spring Boot-based, we can run it from the command line by using $ mvn spring-boot:run from the root of our application, as shown (with its output) in the following example:

The preceding output has three lines that of interest to us here:

The information stored in the TaskRepository is modeled in the TaskExecution class and consists of the following information:

When a task completes, it tries to return an exit code to the OS. If we take a look at our original example, we can see that we are not controlling that aspect of our application. So, if an exception is thrown, the JVM returns a code that may or may not be of any use to you in debugging.

Consequently, Spring Boot provides an interface, ExitCodeExceptionMapper, that lets you map uncaught exceptions to exit codes. Doing so lets you indicate, at the level of exit codes, what went wrong. Also, by mapping exit codes in this manner, Spring Cloud Task records the returned exit code.

If the task terminates with a SIG-INT or a SIG-TERM, the exit code is zero unless otherwise specified within the code.

Spring Cloud Task uses a datasource for storing the results of task executions. By default, we provide an in-memory instance of H2 to provide a simple method of bootstrapping development. However, in a production environment, you probably want to configure your own DataSource.

If your application uses only a single DataSource and that serves as both your business schema and the task repository, all you need to do is provide any DataSource (the easiest way to do so is through Spring Boot’s configuration conventions). This DataSource is automatically used by Spring Cloud Task for the repository.

If your application uses more than one DataSource, you need to configure the task repository with the appropriate DataSource. This customization can be done through an implementation of TaskConfigurer.

One modifiable property of TaskRepository is the table prefix for the task tables. By default, they are all prefaced with TASK_. TASK_EXECUTION and TASK_EXECUTION_PARAMS are two examples. However, there are potential reasons to modify this prefix. If the schema name needs to be prepended to the table names or if more than one set of task tables is needed within the same schema, you must change the table prefix. You can do so by setting the spring.cloud.task.tablePrefix to the prefix you need, as follows:

spring.cloud.task.tablePrefix=yourPrefix

By using the spring.cloud.task.tablePrefix, a user assumes the responsibility to create the task tables that meet both the criteria for the task table schema but with modifications that are required for a user’s business needs. You can utilize the Spring Cloud Task Schema DDL as a guide when creating your own Task DDL as seen here.

In cases where you are creating the task tables and do not wish for Spring Cloud Task to create them at task startup, set the spring.cloud.task.initialize-enabled property to false, as follows:

spring.cloud.task.initialize-enabled=false

It defaults to true.

In some cases, you may want to allow for the time difference between when a task is requested and when the infrastructure actually launches it. Spring Cloud Task lets you create a TaskExecution when the task is requested. Then pass the execution ID of the generated TaskExecution to the task so that it can update the TaskExecution through the task’s lifecycle.

A TaskExecution can be created by calling the createTaskExecution method on an implementation of the TaskRepository that references the datastore that holds the TaskExecution objects.

In order to configure your Task to use a generated TaskExecutionId, add the following property:

spring.cloud.task.executionid=yourtaskId

Spring Cloud Task lets you store an external task ID for each TaskExecution. In order to configure your Task to use a generated TaskExecutionId, add the following property:

spring.cloud.task.external-execution-id=<externalTaskId>

Spring Cloud Task lets you store a parent task ID for each TaskExecution. An example of this would be a task that executes another task or tasks and you want to record which task launched each of the child tasks. In order to configure your Task to set a parent TaskExecutionId add the following property on the child task:

spring.cloud.task.parent-execution-id=<parentExecutionTaskId>

The TaskConfigurer is a strategy interface that lets you customize the way components of Spring Cloud Task are configured. By default, we provide the DefaultTaskConfigurer that provides logical defaults: Map-based in-memory components (useful for development if no DataSource is provided) and JDBC based components (useful if there is a DataSource available).

The TaskConfigurer lets you configure three main components:

You can customize any of the components described in the preceding table by creating a custom implementation of the TaskConfigurer interface. Typically, extending the DefaultTaskConfigurer (which is provided if a TaskConfigurer is not found) and overriding the required getter is sufficient. However, implementing your own from scratch may be required.

TaskExecutionListener lets you register listeners for specific events that occur during the task lifecycle. To do so, create a class that implements the TaskExecutionListener interface. The class that implements the TaskExecutionListener interface is notified of the following events:

Spring Cloud Task also lets you add TaskExecution Listeners to methods within a bean by using the following method annotations:

The following example shows the three annotations in use:

Spring Cloud Task lets you establish that only one task with a given task name can be run at a time. To do so, you need to establish the task name and set spring.cloud.task.single-instance-enabled=true for each task execution. While the first task execution is running, any other time you try to run a task with the same task name and`spring.cloud.task.single-instance-enabled=true`, the task fails with the following error message: Task with name "application" is already running. The default value for spring.cloud.task.single-instance-enabled is false. The following example shows how to set spring.cloud.task.single-instance-enabled to true:

spring.cloud.task.single-instance-enabled=true or false

To use this feature, you must add the following Spring Integration dependencies to your application:

To Enable Task Observations for ApplicationRunner or CommandLineRunner set spring.cloud.task.observation.enabled to true.

An example task application with observations enables using the SimpleMeterRegistry can be found here.

In cases where Spring Cloud Task should not be autoconfigured for an implementation, you can disable Task’s auto configuration. This can be done either by adding the following annotation to your Task application:

You may also disable Task auto configuration by setting the spring.cloud.task.autoconfiguration.enabled property to false.

If the application requires the ApplicationContext to be closed at the completion of a task (all *Runner#run methods have been called and the task repository has been updated), set the property spring.cloud.task.closecontextEnabled to true.

Another case to close the context is when the Task Execution completes however the application does not terminate. In these cases the context is held open because a thread has been allocated (for example: if you are using a TaskExecutor). In these cases set the spring.cloud.task.closecontextEnabled property to true when launching your task. This will close the application’s context once the task is complete. Thus allowing the application to terminate.

Spring Cloud Task integrates with Micrometer and creates observations for the Tasks it executes. To enable Task Observability integration, you must add spring-boot-starter-actuator, your preferred registry implementation (if you want to publish metrics), and micrometer-tracing (if you want to publish tracing data) to your task application. An example maven set of dependencies to enable task observability and metrics using Influx would be:

The term task is frequently used word in the industry. In one such example Spring Boot offers the spring.task while Spring Cloud Task offers the spring.cloud.task properties. This has caused some confusion in the past that these two groups of properties are directly related. However, they represent 2 different set of features offered in the Spring ecosystem.

To prevent the listener from being injected into any batch jobs within the current context, you can disable the autoconfiguration by using standard Spring Boot mechanisms.

To only have the listener injected into particular jobs within the context, override the batchTaskExecutionListenerBeanPostProcessor and provide a list of job bean IDs, as shown in the following example:

By default batch partitions are launched sequentially. However, in some cases this may affect performance as each launch will block until the resource (For example: provisioning a pod in Kubernetes) is provisioned. In these cases you can provide a ThreadPoolTaskExecutor to the DeployerPartitionHandler. This will launch the remote batch partitions based on the configuration of the ThreadPoolTaskExecutor. For example:

To begin, the starter provides a set of properties that let you configure the basics of a Job with one Step:

With the above properties configured, you have a job with a single, chunk-based step. This chunk-based step reads, processes, and writes Map<String, Object> instances as the items. However, the step does not yet do anything. You need to configure an ItemReader, an optional ItemProcessor, and an ItemWriter to give it something to do. To configure one of these, you can either use properties and configure one of the options that has provided autoconfiguration or you can configure your own with the standard Spring configuration mechanisms.

You can read from a queue or topic with AMQP by using the AmqpItemReader. The autoconfiguration for this ItemReader implementation is dependent upon two sets of configuration. The first is the configuration of an AmqpTemplate. You can either configure this yourself or use the autoconfiguration provided by Spring Boot. See the Spring Boot AMQP documentation. Once you have configured the AmqpTemplate, you can enable the batch capabilities to support it by setting the following properties:

For more information, see the AmqpItemReader documentation.

FlatFileItemReader lets you read from flat files (such as CSVs and other file formats). To read from a file, you can provide some components yourself through normal Spring configuration (LineTokenizer, RecordSeparatorPolicy, FieldSetMapper, LineMapper, or SkippedLinesCallback). You can also use the following properties to configure the reader:

See the FlatFileItemReader documentation.

The JdbcCursorItemReader runs a query against a relational database and iterates over the resulting cursor (ResultSet) to provide the resulting items. This autoconfiguration lets you provide a PreparedStatementSetter, a RowMapper, or both. You can also use the following properties to configure a JdbcCursorItemReader:

You can also specify JDBC DataSource specifically for the reader by using the following properties: .JdbcCursorItemReader Properties

See the JdbcCursorItemReader documentation.

Ingesting a partition of data from a Kafka topic is useful and exactly what the KafkaItemReader can do. To configure a KafkaItemReader, two pieces of configuration are required. First, configuring Kafka with Spring Boot’s Kafka autoconfiguration is required (see the Spring Boot Kafka documentation). Once you have configured the Kafka properties from Spring Boot, you can configure the KafkaItemReader itself by setting the following properties:

See the KafkaItemReader documentation.

The advantage of Single Step Batch Processing is that it lets you dynamically select which reader and writer beans to use at runtime when you use the JVM. However, when you use native compilation, you must determine the reader and writer at build time instead of runtime. The following example does so:

To write to a RabbitMQ queue, you need two sets of configuration. First, you need an AmqpTemplate. The easiest way to get this is by using Spring Boot’s RabbitMQ autoconfiguration. See the Spring Boot AMQP documentation.

Once you have configured the AmqpTemplate, you can configure the AmqpItemWriter by setting the following properties:

To write a file as the output of the step, you can configure FlatFileItemWriter. Autoconfiguration accepts components that have been explicitly configured (such as LineAggregator, FieldExtractor, FlatFileHeaderCallback, or a FlatFileFooterCallback) and components that have been configured by setting the following properties specified:

See the FlatFileItemWriter documentation.

To write the output of a step to a relational database, this starter provides the ability to autoconfigure a JdbcBatchItemWriter. The autoconfiguration lets you provide your own ItemPreparedStatementSetter or ItemSqlParameterSourceProvider and configuration options by setting the following properties:

You can also specify JDBC DataSource specifically for the writer by using the following properties: .JdbcBatchItemWriter Properties

See the JdbcBatchItemWriter documentation.

To write step output to a Kafka topic, you need KafkaItemWriter. This starter provides autoconfiguration for a KafkaItemWriter by using facilities from two places. The first is Spring Boot’s Kafka autoconfiguration. (See the Spring Boot Kafka documentation.) Second, this starter lets you configure two properties on the writer.

For more about the configuration options for the KafkaItemWriter, see the KafkaItemWiter documentation.

When using Spring AOT with Single Step Batch Starter you must set the reader and writer name properties at compile time (unless you create a bean(s) for the reader and or writer). To do this you must include the name of the reader and writer that you wish to use as and argument or environment variable in the boot maven plugin or gradle plugin. For example if you wish to enable the FlatFileItemReader and FlatFileItemWriter in Maven it would look like:

To create a stream in Spring Cloud Data Flow, you must first register the Task Sink Application we created. In the following example, we are registering the Processor and Sink sample applications by using the Spring Cloud Data Flow shell:

The following example shows how to create a stream from the Spring Cloud Data Flow shell:

To disable task events, you can set the spring.cloud.task.events.enabled property to false.

One of the options that Spring Cloud Task offers for batch events is the ability to alter the channel to which a specific listener can emit its messages. To do so, use the following configuration: spring.cloud.stream.bindings.<the channel>.destination=<new destination>. For example, if StepExecutionListener needs to emit its messages to another channel called my-step-execution-events instead of the default step-execution-events, you can add the following configuration:

spring.cloud.task.batch.events.step-execution-events-binding-name=my-step-execution-events

To disable the listener functionality for all batch events, use the following configuration:

spring.cloud.task.batch.events.enabled=false

To disable a specific batch event, use the following configuration:

spring.cloud.task.batch.events.<batch event listener>.enabled=false:

The following listing shows individual listeners that you can disable:

By default, batch events have Ordered.LOWEST_PRECEDENCE. To change this value (for example, to 5 ), use the following configuration:

By default Spring Cloud Task uses a sequence table for determining the TASK_EXECUTION_ID for the TASK_EXECUTION table. However, when launching multiple tasks simultaneously while using SQL Server, this can cause a deadlock to occur on the TASK_SEQ table. The resolution is to drop the TASK_EXECUTION_SEQ table and create a sequence using the same name. For example:

Below you can find a list of all metrics declared by this project.

Below you can find a list of all spans declared by this project.