This section goes into more detail about how you can work with Spring Cloud Task Starters as standalone applications or with Spring Cloud Data Flow. It assumes familiarity with general Spring Cloud Task concepts, which can be found in the Spring Cloud Task reference documentation.
Spring Cloud Task Application Starters provide you with predefined Spring Cloud Task applications that you can run independently or with Spring Cloud Data Flow. You can also use the starters as a basis for creating your own applications. They include commonly used tasks that can run as is or be modified to your needs.
As a user of Spring Cloud Task Application Starters you have access to two types of artifacts.
Starters are libraries that contain the complete configuration of a Spring Cloud Task application with a specific role (e.g. an JDBC HDFS that migrates data from a JDBC Repository via sql query to a file on hdfs). Starters are not executable applications, and are intended to be included in other Spring Boot applications.
Prebuilt applications are Spring Boot applications that include the starters. Prebuilt applications are uberjars and include minimal code required to execute standalone.
Note | |
---|---|
Only starters are present in the source code of the project. Prebuilt applications are generated according to the Maven plugin configuration. |
Starters are available as Maven artifacts in the Spring repositories. You can add them as dependencies to your application, as follows:
<dependency> <group>org.springframework.cloud.task.app</group> <artifactId>spring-cloud-starter-task-timestamp</artifactId> <version>1.0.0.BUILD-SNAPSHOT</version> </dependency>
From this, you can infer the coordinates for other starters found in this guide.
While the version may vary, the group will always remain org.springframework.cloud.task.app
and the artifact id follows the naming convention spring-cloud-starter-task-<functionality>
.
Prebuilt applications are available as Maven artifacts too.
It is not encouraged to use them directly as dependencies, as starters should be used instead.
Following the typical Maven <group>:<artifactId>:<version>
convention, they can be referenced for example as:
org.springframework.cloud.task.app:timestamp-task:1.0.0.BUILD-SNAPSHOT
Just as with the starters, you can infer the coordinates for other prebuilt applications found in the guide.
The group will be always org.springframework.cloud.task.app
.
The version may vary.
The artifact id follows the format <functionality>-task
.
Docker
The Docker versions of the applications are available in Docker Hub, at hub.docker.com/r/springcloudtask/. Naming and versioning follows the same general conventions as Maven, e.g.
docker pull springcloudtask/timestamp-task
will pull the latest Docker image of the timestamp task.
You can also build the project and generate the artifacts (including the prebuilt applications) on your own. This is useful if you want to deploy the artifacts locally, for example for adding a new starter.
First, you need to generate the prebuilt applications. This is done by running the application generation Maven plugin. You can do so by simply invoking the corresponding script in the root of the project.
./generate.sh
Then build the applications:
cd apps mvn clean install
For the each of the prebuilt applications, the script will generate the following items:
pom.xml
file with the required dependenciesmain
method of the application and imports the predefined configurationIn this section we will describe how to create your own application.
Spring Cloud Task Application Starters consist of regular Spring Cloud Task applications with some additional conventions that facilitate generating prebuilt applications. Sometimes, your solution may require additional applications that are not in the scope of Spring Cloud Task Application Starters, or require additional tweaks and enhancements. In this section we will show you how to create custom applications that can be part of your solution, along with Spring Cloud Task application starters. You have the following options:
If you want to add your own custom applications to your solution, you can simply create a new Spring Cloud Task project and run it the same way as the applications provided by Spring Cloud Task Application Starters, independently or via Spring Cloud Data Flow. The process is described in the Getting Started Guide of Spring Cloud Task.
You can also reuse the starters provided by Spring Cloud Task Application Starters to create custom components, enriching the behavior of the application. For example, you can add a special behavior to your jdbc hdfs task, to do some post processing following the migration of the data. As a reminder, this involves:
<dependencies> <!- other dependencies --> <dependencies> <dependency> <groupId>org.springframework.cloud.task.app</groupId> <artifactId>spring-cloud-starter-task-timestamp</artifactId> </dependency> </dependencies>
package org.springframework.cloud.task.app.timestamp; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.context.annotation.Import; @SpringBootApplication @Import(org.springframework.cloud.task.app.timestamp.TimestampTaskConfiguration.class) public class TimestampTaskApplication { public static void main(String[] args) { SpringApplication.run(TimestampTaskApplication.class, args); } }
After doing so, you can simply add the additional configuration for the extra features of your application.
In this section, we will explain how to develop a custom task application and then generate maven and docker artifacts for it using the existing tooling provided by the spring cloud task app starter infrastructure. For explanation purposes, we will assume that we are creating a new task application for a technology named foobar.
Please look into the existing starters for how to design and structure a new one. Ensure that you name the
main @Configuration
class of your starter as FoobarTaskConfiguration
as this is the default convention used by
the app generation later. The default package for the class with @Configuration
is org.springfamework.cloud.task.app.foobar
.
If you have a different class/package name, see below for how to override that in the app generator.
The technology name for which the app starter is created can be a hyphenated stream of strings such as in spark-client
This starter module for this needs to be spring-cloud-starter-task-spark-client
.
The starters in spring-cloud-task-app-starters
are slightly different from the other starters in spring-boot and
spring-cloud in that here we don’t provide a way to auto configure any configuration through spring factories mechanism.
Rather, we delegate this responsibility to the maven plugin that is generating the binder based apps. Therefore, you don’t
have to provide a spring.factories file that lists all your configuration classes.
./mvnw clean install -pl :spring-cloud-starter-task-foobar
spring-cloud-task-app-dependencies
bill of material (BOM) in the
dependecy management section. For example,<dependencyManagement> ... ... <dependency> <groupId>org.springframework.cloud.task.app</groupId> <artifactId>spring-cloud-starter-task-foobar</artifactId> <version>1.0.0.BUILD-SNAPSHOT</version> </dependency> ... ...
./mvnw clean install -pl :spring-cloud-task-app-dependencies
spring-cloud-task-app-generator
module and start editing as below.The minimal configuration needed to generate the app is to add to plugin configuration in spring-cloud-task-app-generator/pom.xml. There are other plugin options that customize the generated applications which are described in the plugin documentation (github.com/spring-cloud/spring-cloud-stream-app-maven-plugin). A few plugin features are described below.
<generatedApps> .... <foobar-task /> .... </generatedApps>
More information about the maven plugin used above can be found here: github.com/spring-cloud/spring-cloud-stream-app-maven-plugin
If you did not follow the default convention expected by the plugin of where it is looking for the main configuration
class, which is org.springfamework.cloud.task.app.foobar.FoobarTaskConfiguration
, you can override that in
the configuration for the plugin. For example, if your main configuration class is foo.bar.SpecialFooBarTaskConfiguration.class
,
this is how you can tell the plugin to override the default.
<foobar-task> <autoConfigClass>foo.bar.SpecialFooBarTaskConfiguration.class</autoConfigClass> </foobar-task>
./generateApps.sh
This will generate the foobar task app in a directory named apps
at the root of the repository.
If you want to change the location where the apps are generated, for instance /tmp/task-apps, you can do it in the
configuration section of the plugin.
<configuration> ... <generatedProjectHome>/tmp/task-apps</generatedProjectHome> ... </configuration
If you have an artifact that is only available through a private internal maven repository (may be an enterprise wide Nexus repo that you use globally across teams), and you need that for your app, you can define that as part of the maven plugin configuration.
For example,
<configuration> ... <extraRepositories> <repository> <id>private-internal-nexus</id> <url>.../</url> <name>...</name> <snapshotEnabled>...</snapshotEnabled> </repository> </extraRepositories> </configuration>
Then you can define this as part of your app tag:
<foobar-task> <extraRepositories> <private-internal-nexus /> </extraRepositories> </foobar-task>
apps
at the root of the repository by default, unless you changed
it elsewhere as described above).Here you will see foobar-task
along with all the other out of the box apps that is generated.
If you only care about the foobar-task apps and nothing else, you can cd into that directory
and import it directly into your IDE of choice. Each of them is a self contained spring boot application project.
For all the generated apps, the parent is spring-boot-starter-parent
as is required by Spring Initializr, the library
used under the hood to generate the apps.
You can cd into these custom foobar-task directories and do the following to build the apps:
cd foobar-task
mvn clean install
This would install the foobar-task into your local maven cache (~/.m2 by default).
The app generation phase adds an integration test to the app project that ensures all the spring
components and contexts are loaded properly. However, these tests are not run by default when you do a mvn install
.
You can force the running of these tests by doing the following:
mvn clean install -DskipTests=false
target
directories of the respective apps and also as
maven artifacts in your local maven repository. Go to the target
directory and run the following:java -jar foobar-task.jar
It should start the application up.
mvn clean package docker:build
This creates the docker image under the target/docker/springcloudtask
directory. Please ensure that the Docker
container is up and running and DOCKER_HOST environment variable is properly set before you try docker:build
.
All the generated apps from the repository are uploaded to Docker Hub
However, for a custom app that you build, this won’t be uploaded to docker hub under springcloudtask
repository.
If you think that there is a general need for this app, you should contribute this starter to the main repository
and upon review, this app then can be uploaded to the above location in docker hub.
If you still need to push this to docker hub under a different repository you can take the following steps.
Go to the pom.xml of the generated app [ example - foobar-task/pom.xml
]
Search for springcloudtask
. Replace with your repository name.
Then do this:
mvn clean package docker:build docker:push -Ddocker.username=[provide your username] -Ddocker.password=[provide password]
This would upload the docker image to the docker hub in your custom repository.