Skipper is a server that allows you to discover Spring Boot applications and manage their lifecycle on multiple Cloud Platforms.
Applications in Skipper are bundled as packages which contain templated configuration files. They also contain an optional values file that contains default values using to fill in template placeholders. You can find out more about the format of the package .zip file in Skipper’s documentation on Packages.
Skipper’s templated configuration files contain placeholders for application properties, application version, and deployment properties.
Package .zip files are uploaded to Skipper and stored in a package repository.
Skipper’s package repository is analogous to those found in tools such as apt-get or brew.
You can override template values when installing or upgrading a package. Skipper orchestrates the upgrade/rollback procedure of applications between different versions, taking the minimal set of actions to bring the system to the desired state. For example, if only one application in a stream has been updated, only that single application is deployed with a new version and the old version undeployed. An application is considered different when upgrading if any of it’s application properties, deployment properties (excluding count), or application version (e.g. 1.0.0.RELEASE) is different from the currently installed application.
Spring Cloud Data Flow is integrated with Skipper by generating a Skipper package when deploying a Stream. The generated package name is the same name as the Stream. The generated package is uploaded to Skipper’s package repository and Data Flow then instructs Skipper to install the package that corresponds to the Stream. Subsequent commands to upgrade and rollback the applications within the Stream are passed through to Skipper after some validation checks are performed by Data Flow.
Skipper extends the Register a Stream App lifecycle with support of multi-versioned stream applications. This allows to upgrade or rollback those applications at runtime using the deployment properties.
Register a versioned stream application using the app register command. You must provide a unique name, application type, and a URI that can be resolved to the app artifact.
For the type, specify "source", "processor", or "sink". The version is resolved from the URI. Here are a few examples:
dataflow:>app register --name mysource --type source --uri maven://com.example:mysource:0.0.1 dataflow:>app register --name mysource --type source --uri maven://com.example:mysource:0.0.2 dataflow:>app register --name mysource --type source --uri maven://com.example:mysource:0.0.3 dataflow:>app list --id source:mysource ╔══════════════════╤═════════╤════╤════╗ ║ source │processor│sink│task║ ╠══════════════════╪═════════╪════╪════╣ ║> mysource-0.0.1 <│ │ │ ║ ║mysource-0.0.2 │ │ │ ║ ║mysource-0.0.3 │ │ │ ║ ╚══════════════════╧═════════╧════╧════╝
The application URI should conform to one the following schema formats:
- maven schema
maven://<groupId>:<artifactId>[:<extension>[:<classifier>]]:<version>
- http schema
http://<web-path>/<artifactName>-<version>.jar
- file schema
file:///<local-path>/<artifactName>-<version>.jar
- docker schema
docker:<docker-image-path>/<imageName>:<version>
![[Note]](images/note.png) | Note |
|---|---|
The URI |
Multiple versions can be registered for the same applications (e.g. same name and type) but only one can be set as default. The default version is used for deploying Streams.
The first time an application is registered it will be marked as default. The default application version can be altered with the app default command:
dataflow:>app default --id source:mysource --version 0.0.2 dataflow:>app list --id source:mysource ╔══════════════════╤═════════╤════╤════╗ ║ source │processor│sink│task║ ╠══════════════════╪═════════╪════╪════╣ ║mysource-0.0.1 │ │ │ ║ ║> mysource-0.0.2 <│ │ │ ║ ║mysource-0.0.3 │ │ │ ║ ╚══════════════════╧═════════╧════╧════╝
The app list --id <type:name> command lists all versions for a given stream application.
The app unregister command has an optional --version parameter to specify the app version to unregister.
dataflow:>app unregister --name mysource --type source --version 0.0.1 dataflow:>app list --id source:mysource ╔══════════════════╤═════════╤════╤════╗ ║ source │processor│sink│task║ ╠══════════════════╪═════════╪════╪════╣ ║> mysource-0.0.2 <│ │ │ ║ ║mysource-0.0.3 │ │ │ ║ ╚══════════════════╧═════════╧════╧════╝
If a --version is not specified, the default version is unregistered.
| Note |
|---|---|
All applications in a stream should have a default version set for the stream to be deployed.
Otherwise they will be treated as unregistered application during the deployment.
Use the |
app default --id source:mysource --version 0.0.3 dataflow:>app list --id source:mysource ╔══════════════════╤═════════╤════╤════╗ ║ source │processor│sink│task║ ╠══════════════════╪═════════╪════╪════╣ ║mysource-0.0.2 │ │ │ ║ ║> mysource-0.0.3 <│ │ │ ║ ╚══════════════════╧═════════╧════╧════╝
The stream deploy necessitates default app versions to be set.
The stream update and stream rollback commands though can use all (default and non-default) registered app versions.
dataflow:>stream create foo --definition "mysource | log"This will create stream using the default mysource version (0.0.3). Then we can update the version to 0.0.2 like this:
dataflow:>stream update foo --properties version.mysource=0.0.2
![[Important]](images/important.png) | Important |
|---|---|
Only pre-registered applications can be used to |
An attempt to update the mysource to version 0.0.1 (not registered) will fail!
You create and deploy a stream as follows:
dataflow:> stream create --name httptest --definition "http --server.port=9000 | log" --deployIf you want to pass deployment properties, you can create and deploy a stream in two steps:
dataflow:> stream create --name httptest --definition "http --server.port=9000 | log"
dataflow:> stream deploy --name httptestThe command stream info shows useful information about the stream including the deployment properties.
dataflow:>stream info httptest ╔══════════════════════════════╤══════════════════════════════╤══════════════════════════════╗ ║ Name │ DSL │ Status ║ ╠══════════════════════════════╪══════════════════════════════╪══════════════════════════════╣ ║httptest │http --server.port=9000 | log │deploying ║ ╚══════════════════════════════╧══════════════════════════════╧══════════════════════════════╝ Stream Deployment properties: { "log" : { "spring.cloud.deployer.indexed" : "true", "spring.cloud.deployer.group" : "httptest", "maven://org.springframework.cloud.stream.app:log-sink-rabbit" : "1.1.0.RELEASE" }, "http" : { "spring.cloud.deployer.group" : "httptest", "maven://org.springframework.cloud.stream.app:http-source-rabbit" : "1.1.0.RELEASE" } }
There is an important optional command argument to the stream deploy command, which is --platformName.
Skipper can be configured to deploy to multiple platforms.
Skipper is pre-configured with a platform named default which will deploys applications to the local machine where Skipper is running.
The default value of the command line argument --platformName is default.
If you are commonly deploying to one platform, when installing Skipper you can override the configuration of the default platform.
Otherwise, specify the platformName to one of the values returned by the command stream platform-list
| Note |
|---|---|
In future releases, only the local Data Flow server will be configured with the |
To update the stream, use the command stream update which takes as a command argument either --properties or --propertiesFile.
You can pass in values to these command arguments in the same format as when deploy the stream with or without Skipper.
There is an important new top level prefix available when using Skipper, which is version.
If the Stream http | log was deployed, and the version of log which registered at the time of deployment was 1.1.0.RELEASE, the following command will update the Stream to use the 1.2.0.RELEASE of the log application.
Before updating the stream with the specific version of the app, we need to make sure that the app is registered with that version.
dataflow:>app register --name log --type sink --uri maven://org.springframework.cloud.stream.app:log-sink-rabbit:1.2.0.RELEASE Successfully registered application 'sink:log'
dataflow:>stream update --name httptest --properties version.log=1.2.0.RELEASE
| Important |
|---|---|
Only pre-registered application versions can be used to |
To verify the deployment properties and the updated version, we can use stream info
dataflow:>stream info httptest ╔══════════════════════════════╤══════════════════════════════╤══════════════════════════════╗ ║ Name │ DSL │ Status ║ ╠══════════════════════════════╪══════════════════════════════╪══════════════════════════════╣ ║httptest │http --server.port=9000 | log │deploying ║ ╚══════════════════════════════╧══════════════════════════════╧══════════════════════════════╝ Stream Deployment properties: { "log" : { "spring.cloud.deployer.indexed" : "true", "spring.cloud.deployer.count" : "1", "spring.cloud.deployer.group" : "httptest", "maven://org.springframework.cloud.stream.app:log-sink-rabbit" : "1.2.0.RELEASE" }, "http" : { "spring.cloud.deployer.group" : "httptest", "maven://org.springframework.cloud.stream.app:http-source-rabbit" : "1.1.0.RELEASE" } }
Skipper keeps a history of the Streams that were deployed.
After updating a Stream, there will be a second version of the stream.
You can query for the history of the versions using the command stream history --name <name-of-stream>.
dataflow:>stream history --name httptest ╔═══════╤════════════════════════════╤════════╤════════════╤═══════════════╤════════════════╗ ║Version│ Last updated │ Status │Package Name│Package Version│ Description ║ ╠═══════╪════════════════════════════╪════════╪════════════╪═══════════════╪════════════════╣ ║2 │Mon Nov 27 22:41:16 EST 2017│DEPLOYED│httptest │1.0.0 │Upgrade complete║ ║1 │Mon Nov 27 22:40:41 EST 2017│DELETED │httptest │1.0.0 │Delete complete ║ ╚═══════╧════════════════════════════╧════════╧════════════╧═══════════════╧════════════════╝
Skipper keeps an "manifest" of the all the applications, their application properties and deployment properties after all values have been substituted.
This represents the final state of what was deployed to the platform.
You can view the manifest for any of the versions of a Stream using the command stream manifest --name <name-of-stream> --releaseVersion <optional-version>
If the --releaseVersion is not specified, the manifest for the last version is returned.
dataflow:>stream manifest --name httptest --- # Source: log.yml apiVersion: skipper.spring.io/v1 kind: SpringCloudDeployerApplication metadata: name: log spec: resource: maven://org.springframework.cloud.stream.app:log-sink-rabbit version: 1.2.0.RELEASE applicationProperties: spring.metrics.export.triggers.application.includes: integration** spring.cloud.dataflow.stream.app.label: log spring.cloud.stream.metrics.key: httptest.log.${spring.cloud.application.guid} spring.cloud.stream.bindings.input.group: httptest spring.cloud.stream.metrics.properties: spring.application.name,spring.application.index,spring.cloud.application.*,spring.cloud.dataflow.* spring.cloud.dataflow.stream.name: httptest spring.cloud.dataflow.stream.app.type: sink spring.cloud.stream.bindings.input.destination: httptest.http deploymentProperties: spring.cloud.deployer.indexed: true spring.cloud.deployer.group: httptest spring.cloud.deployer.count: 1 --- # Source: http.yml apiVersion: skipper.spring.io/v1 kind: SpringCloudDeployerApplication metadata: name: http spec: resource: maven://org.springframework.cloud.stream.app:http-source-rabbit version: 1.2.0.RELEASE applicationProperties: spring.metrics.export.triggers.application.includes: integration** spring.cloud.dataflow.stream.app.label: http spring.cloud.stream.metrics.key: httptest.http.${spring.cloud.application.guid} spring.cloud.stream.bindings.output.producer.requiredGroups: httptest spring.cloud.stream.metrics.properties: spring.application.name,spring.application.index,spring.cloud.application.*,spring.cloud.dataflow.* server.port: 9000 spring.cloud.stream.bindings.output.destination: httptest.http spring.cloud.dataflow.stream.name: httptest spring.cloud.dataflow.stream.app.type: source deploymentProperties: spring.cloud.deployer.group: httptest
The majority of the deployment and application properties were set by Data Flow in order to enable the applications to talk to each other and sending application metrics with identifying labels.
You can rollback to a previous version of the Stream using the command stream rollback.
dataflow:>stream rollback --name httptest
There is an optional --releaseVersion command argument which is the version of the Stream.
If not specified, the rollback goes to the previous stream version.
The application count is a dynamic property of the system. If due to scaling at runtime, the application to be upgraded has 5 instances running, then 5 instances of the upgraded application will be deployed.
Skipper has a simple 'red/black' upgrade strategy. It deploys the new version of the applications, as many instances as the currently running version, and checks the /health endpoint of the application.
If the health of the new application is good, then the previous application is undeployed.
If the health of the new application is bad, then all new applications are undeployed and the upgrade is considered not successful.
The upgrade strategy is not a rolling upgrade, so if 5 applications of the application to upgrade are runningn, then in a sunny day scenario, 5 of the new applications will also be running before the older version is undeployed. Future versions of Skipper will support rolling upgrades and other types of checks, e.g. manual, to continue to upgrade process.