Register a Stream App with the App Registry using the Spring Cloud Data Flow Shell
app register
command. You must provide a unique name, application type, and a URI that can be
resolved to the app artifact. For the type, specify "source", "processor", or "sink".
Here are a few examples:
dataflow:>app register --name mysource --type source --uri maven://com.example:mysource:0.0.1-SNAPSHOT dataflow:>app register --name myprocessor --type processor --uri file:///Users/example/myprocessor-1.2.3.jar dataflow:>app register --name mysink --type sink --uri http://example.com/mysink-2.0.1.jar
When providing a URI with the maven
scheme, the format should conform to the following:
maven://<groupId>:<artifactId>[:<extension>[:<classifier>]]:<version>
For example, if you would like to register the snapshot versions of the http
and log
applications built with the RabbitMQ binder, you could do the following:
dataflow:>app register --name http --type source --uri maven://org.springframework.cloud.stream.app:http-source-rabbit:1.1.2.BUILD-SNAPSHOT dataflow:>app register --name log --type sink --uri maven://org.springframework.cloud.stream.app:log-sink-rabbit:1.1.2.BUILD-SNAPSHOT
If you would like to register multiple apps at one time, you can store them in a properties file
where the keys are formatted as <type>.<name>
and the values are the URIs.
For example, if you would like to register the snapshot versions of the http
and log
applications built with the RabbitMQ binder, you could have the following in a properties file [eg: stream-apps.properties]:
source.http=maven://org.springframework.cloud.stream.app:http-source-rabbit:1.1.2.BUILD-SNAPSHOT sink.log=maven://org.springframework.cloud.stream.app:log-sink-rabbit:1.1.2.BUILD-SNAPSHOT
Then to import the apps in bulk, use the app import
command and provide the location of the properties file via --uri
:
dataflow:>app import --uri file:///<YOUR_FILE_LOCATION>/stream-apps.properties
For convenience, we have the static files with application-URIs (for both maven and docker) available for all the out-of-the-box stream and task/batch app-starters. You can point to this file and import all the application-URIs in bulk. Otherwise, as explained in previous paragraphs, you can register them individually or have your own custom property file with only the required application-URIs in it. It is recommended, however, to have a "focused" list of desired application-URIs in a custom property file.
List of available Stream Application Starters:
Artifact Type | Stable Release | SNAPSHOT Release |
---|---|---|
RabbitMQ + Maven | http://bit.ly/Bacon-BUILD-SNAPSHOT-stream-applications-rabbit-maven | |
RabbitMQ + Docker | http://bit.ly/Avogadro-SR1-stream-applications-rabbit-docker | N/A ] |
Kafka 0.9 + Maven | http://bit.ly/Avogadro-SR1-stream-applications-kafka-09-maven | http://bit.ly/Bacon-BUILD-SNAPSHOT-stream-applications-kafka-09-maven |
Kafka 0.9 + Docker | http://bit.ly/Avogadro-SR1-stream-applications-kafka-09-docker | N/A ] |
Kafka 0.10 + Maven | http://bit.ly/Avogadro-SR1-stream-applications-kafka-10-maven | http://bit.ly/Bacon-BUILD-SNAPSHOT-stream-applications-kafka-10-maven |
Kafka 0.10 + Docker | http://bit.ly/Avogadro-SR1-stream-applications-kafka-10-docker | N/A ] |
List of available Task Application Starters:
Artifact Type | Stable Release | SNAPSHOT Release |
---|---|---|
Maven | http://bit.ly/Belmont-BUILD-SNAPSHOT-task-applications-maven | |
Docker | N/A ] |
You can find more information about the available task starters look the Task App Starters Project Page and related reference documentation. For more information about the available stream starters look at the Stream App Starters Project Page and related reference documentation.
As an example, if you would like to register all out-of-the-box stream applications built with the RabbitMQ binder in bulk, you can with the following command.
dataflow:>app import --uri http://bit.ly/Avogadro-SR1-stream-applications-rabbit-maven
You can also pass the --local
option (which is TRUE by default) to indicate whether the
properties file location should be resolved within the shell process itself. If the location should
be resolved from the Data Flow Server process, specify --local false
.
When using either app register
or app import
, if a stream app is already registered with
the provided name and type, it will not be overridden by default. If you would like to override the
pre-existing stream app, then include the --force
option.
Note | |
---|---|
In some cases the Resource is resolved on the server side, whereas in others the URI will be passed to a runtime container instance where it is resolved. Consult the specific documentation of each Data Flow Server for more detail. |
Stream applications are Spring Boot applications which are aware of many Section 37.1, “Common application properties”, e.g. server.port
but also families of properties such as those with the prefix spring.jmx
and logging
. When creating your own application it is desirable to whitelist properties so that the shell and the UI can display them first as primary properties when presenting options via TAB completion or in drop-down boxes.
To whitelist application properties create a file named spring-configuration-metadata-whitelist.properties
in the META-INF
resource directory. There are two property keys that can be used inside this file. The first key is named configuration-properties.classes
. The value is a comma separated list of fully qualified @ConfigurationProperty
class names. The second key is configuration-properties.names
whose value is a comma separated list of property names. This can contain the full name of property, such as server.port
or a partial name to whitelist a category of property names, e.g. spring.jmx
.
The Spring Cloud Stream application starters are a good place to look for examples of usage. Here is a simple example of the file sink’s spring-configuration-metadata-whitelist.properties
file
configuration-properties.classes=org.springframework.cloud.stream.app.file.sink.FileSinkProperties
If we also wanted to add server.port
to be white listed, then it would look like this:
configuration-properties.classes=org.springframework.cloud.stream.app.file.sink.FileSinkProperties configuration-properties.names=server.port
Important | |
---|---|
Make sure to add 'spring-boot-configuration-processor' as an optional dependency to generate configuration metadata file for the properties. <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-configuration-processor</artifactId> <optional>true</optional> </dependency> |