Spring Cloud Data Flow allows a user to create a directed graph where each node of the graph is a task application. This is done by using the DSL for composed tasks. A composed task can be created via the RESTful API, the Spring Cloud Data Flow Shell, or the Spring Cloud Data Flow UI.
Composed tasks are executed via a task application called the Composed Task Runner.
Out of the box the Composed Task Runner application is not registered with Spring Cloud Data Flow. So, to launch composed tasks we must first register the Composed Task Runner as an application with Spring Cloud Data Flow as follows:
app register --name composed-task-runner --type task --uri maven://org.springframework.cloud.task.app:composedtaskrunner-task:<DESIRED_VERSION>
You can also configure Spring Cloud Data Flow to use a different task definition
name for the composed task runner. This can be done by setting the
spring.cloud.dataflow.task.composedTaskRunnerName
property to the name
of your choice. You can then register the composed task runner application with
the name you set using that property.
The Composed Task Runner application has a dataflow.server.uri
property that is used for validation and for launching child tasks. This defaults
to localhost:9393
. If you run a distributed Spring Cloud Data Flow server, like you would do if you deploy the server on Cloud Foundry,
YARN or Kubernetes, then you need to provide the URI that can be used to access the server. You can either provide this dataflow.server.uri
property for the Composed Task Runner application when launching a composed task, or you can provide a spring.cloud.dataflow.server.uri
property
for the Spring Cloud Data Flow server when it is started. For the latter case the dataflow.server.uri
Composed Task Runner application property
will be automatically set when a composed task is launched.
The DSL for the composed tasks is used when creating a task definition via the task create command. For example:
dataflow:> app register --name timestamp --type task --uri maven://org.springframework.cloud.task.app:timestamp-task:<DESIRED_VERSION> dataflow:> app register --name mytaskapp --type task --uri file:///home/tasks/mytask.jar dataflow:> task create my-composed-task --definition "mytaskapp && timestamp" dataflow:> task launch my-composed-task
In the example above we assume that the applications to be used by our composed task have not been registered yet. So the first two steps we register two task applications. We then create our composed task definition by using the task create command. The composed task DSL in the example above will, when launched, execute mytaskapp and then execute the timestamp application.
But before we launch the my-composed-task definition, we can view what Spring Cloud Data Flow generated for us. This can be done by executing the task list command.
dataflow:>task list ╔══════════════════════════╤══════════════════════╤═══════════╗ ║ Task Name │ Task Definition │Task Status║ ╠══════════════════════════╪══════════════════════╪═══════════╣ ║my-composed-task │mytaskapp && timestamp│unknown ║ ║my-composed-task-mytaskapp│mytaskapp │unknown ║ ║my-composed-task-timestamp│timestamp │unknown ║ ╚══════════════════════════╧══════════════════════╧═══════════╝
Spring Cloud Data Flow created three task definitions, one for each of the
applications that comprises our composed task (my-composed-task-mytaskapp
and
my-composed-task-timestamp
) as well as the composed task (my-composed-task
)
definition. We also see that each of the generated
names for the child tasks is comprised of the name of the composed task and
the name of the application separated by a dash -
. i.e. my-composed-task -
mytaskapp.
Launching a composed task is done the same way as launching a stand-alone task. i.e.
task launch my-composed-task
Once the task is launched and assuming all the tasks complete successfully you will
see three task executions when executing a task execution list
. For example:
dataflow:>task execution list ╔══════════════════════════╤═══╤════════════════════════════╤════════════════════════════╤═════════╗ ║ Task Name │ID │ Start Time │ End Time │Exit Code║ ╠══════════════════════════╪═══╪════════════════════════════╪════════════════════════════╪═════════╣ ║my-composed-task-timestamp│713│Wed Apr 12 16:43:07 EDT 2017│Wed Apr 12 16:43:07 EDT 2017│0 ║ ║my-composed-task-mytaskapp│712│Wed Apr 12 16:42:57 EDT 2017│Wed Apr 12 16:42:57 EDT 2017│0 ║ ║my-composed-task │711│Wed Apr 12 16:42:55 EDT 2017│Wed Apr 12 16:43:15 EDT 2017│0 ║ ╚══════════════════════════╧═══╧════════════════════════════╧════════════════════════════╧═════════╝
In the example above we see that my-compose-task launched and it also launched
the other tasks in sequential order and all of them executed successfully with
"Exit Code" as 0
.
The following list shows how the Exit Status will be set for each step (task) contained in the composed task following each step execution.
TaskExecution
has an ExitMessage
that will be used as the ExitStatus
ExitMessage
is present and the ExitCode
is set to zero then the ExitStatus
for the step will be COMPLETED
.ExitMessage
is present and the ExitCode
is set to any non zero number
then the ExitStatus
for the step will be FAILED
.The same command used to destroy a stand-alone task is the same as destroying a composed task. The only difference is that destroying a composed task will also destroy the child tasks associated with it. For example
dataflow:>task list ╔══════════════════════════╤══════════════════════╤═══════════╗ ║ Task Name │ Task Definition │Task Status║ ╠══════════════════════════╪══════════════════════╪═══════════╣ ║my-composed-task │mytaskapp && timestamp│COMPLETED ║ ║my-composed-task-mytaskapp│mytaskapp │COMPLETED ║ ║my-composed-task-timestamp│timestamp │COMPLETED ║ ╚══════════════════════════╧══════════════════════╧═══════════╝ ... dataflow:>task destroy my-composed-task dataflow:>task list ╔═════════╤═══════════════╤═══════════╗ ║Task Name│Task Definition│Task Status║ ╚═════════╧═══════════════╧═══════════╝
In cases where a composed task execution needs to be stopped. This can be done via the:
The composed task run will be stopped
when the currently running child task completes. The step associated with the
child task that was running at the time that the composed task was stopped will
be marked as STOPPED
as well as the composed task job execution.
In cases where a composed task fails during execution and the status of the
composed task is FAILED
then the task can be restarted. This can be done
via the:
Note | |
---|---|
Restarting a Composed Task job that has been stopped (via the
Spring Cloud Data Flow Dashboard or RESTful API), will relaunch the
|