13. DSL Syntax

In the examples above, we connected a source to a sink using the pipe symbol |. You can also pass parameters to the source and sink configurations. The parameter names will depend on the individual module implementations, but as an example, the http source module exposes a server.port setting which allows you to change the data ingestion port from the default value. To create the stream using port 8000, we would use

dataflow:> stream create --definition "http --server.port=8000 | log" --name myhttpstream

The shell provides tab completion for module parameters and also the shell command module info provides some additional documentation. For more comprehensive documentation on module parameters, please see the Modules chapter.

13.1 Register a Stream App

Register a Stream App with the App Registry using the Spring Cloud Data Flow Shell module register command. You must provide a unique name and a URI that can be resolved to the app artifact. For the type, specify "source", "processor", or "sink". Here are a few examples:

dataflow:>module register --name mysource --type source --uri maven://com.example:mysource:0.0.1-SNAPSHOT

dataflow:>module register --name myprocessor --type processor --uri file:///Users/example/myprocessor-1.2.3.jar

dataflow:>module register --name mysink --type sink --uri http://example.com/mysink-2.0.1.jar

When providing a URI with the maven scheme, the format should conform to the following:

maven://<groupId>:<artifactId>[:<extension>[:<classifier>]]:<version>

If you would like to register multiple apps at one time, you can store them in a properties file where the keys are formatted as <type>.<name> and the values are the URIs. For example, this would be a valid properties file:

source.foo=file:///tmp/foo.jar
sink.bar=file:///tmp/bar.jar

Then use the module import command and provide the location of the properties file via --uri:

module import --uri file:///tmp/stream-apps.properties

You can also pass the --local option (which is TRUE by default) to indicate whether the properties file location should be resolved within the shell process itself. If the location should be resolved from the Data Flow Server process, specify --local false.

When using either module register or module import, if a stream app is already registered with the provided name and type, it will not be overridden by default. If you would like to override the pre-existing stream app, then include the --force option.

[Note]Note

In some cases the Resource is resolved on the server side, whereas in others the URI will be passed to a runtime container instance where it is resolved. Consult the specific documentation of each Data Flow Server for more detail.