By default, the Data Flow server is unsecured and runs on an unencrypted HTTP connection. You can secure your REST endpoints, as well as the Data Flow Dashboard by enabling HTTPS and requiring clients to authenticate using either:
NOTE: By default, the REST endpoints (administration, management and health), as well as the Dashboard UI do not require authenticated access.
By default, the dashboard, management, and health endpoints use HTTP as a transport.
You can switch to HTTPS easily, by adding a certificate to your configuration in
application.yml
.
server: port: 8443 ssl: key-alias: yourKeyAlias key-store: path/to/keystore key-store-password: yourKeyStorePassword key-password: yourKeyPassword trust-store: path/to/trust-store trust-store-password: yourTrustStorePassword
As the default port is | |
The alias (or name) under which the key is stored in the keystore. | |
The path to the keystore file. Classpath resources may also be specified, by using the classpath prefix: | |
The password of the keystore. | |
The password of the key. | |
The path to the truststore file. Classpath resources may also be specified, by using the classpath prefix: | |
The password of the trust store. |
Note | |
---|---|
If HTTPS is enabled, it will completely replace HTTP as the protocol over which the REST endpoints and the Data Flow Dashboard interact. Plain HTTP requests will fail - therefore, make sure that you configure your Shell accordingly. |
For testing purposes or during development it might be convenient to create self-signed certificates. To get started, execute the following command to create a certificate:
$ keytool -genkey -alias dataflow -keyalg RSA -keystore dataflow.keystore \ -validity 3650 -storetype JKS \ -dname "CN=localhost, OU=Spring, O=Pivotal, L=Kailua-Kona, ST=HI, C=US" -keypass dataflow -storepass dataflow
CN is the only important parameter here. It should match the domain you are trying to access, e.g. |
Then add the following to your application.yml
file:
server: port: 8443 ssl: enabled: true key-alias: dataflow key-store: "/your/path/to/dataflow.keystore" key-store-type: jks key-store-password: dataflow key-password: dataflow
This is all that’s needed for the Data Flow Server. Once you start the server, you should be able to access it via https://localhost:8443/. As this is a self-signed certificate, you will hit a warning in your browser, that you need to ignore.
By default self-signed certificates are an issue for the Shell and additional steps are necessary to make the Shell work with self-signed certificates. Two options are available:
Add the self-signed certificate to the JVM truststore
In order to use the JVM truststore option, we need to export the previously created certificate from the keystore:
$ keytool -export -alias dataflow -keystore dataflow.keystore -file dataflow_cert -storepass dataflow
Next, we need to create a truststore which the Shell will use:
$ keytool -importcert -keystore dataflow.truststore -alias dataflow -storepass dataflow -file dataflow_cert -noprompt
Now, you are ready to launch the Data Flow Shell using the following JVM arguments:
$ java -Djavax.net.ssl.trustStorePassword=dataflow \ -Djavax.net.ssl.trustStore=/path/to/dataflow.truststore \ -Djavax.net.ssl.trustStoreType=jks \ -jar spring-cloud-dataflow-shell-1.2.0.M1.jar
Tip | |
---|---|
In case you run into trouble establishing a connection via SSL, you can enable additional
logging by using and setting the |
Don’t forget to target the Data Flow Server with:
dataflow:> dataflow config server https://localhost:8443/
Skip Certificate Validation
Alternatively, you can also bypass the certification validation by providing the
optional command-line parameter --dataflow.skip-ssl-validation=true
.
Using this command-line parameter, the shell will accept any (self-signed) SSL certificate.
Warning | |
---|---|
If possible you should avoid using this option. Disabling the trust manager defeats the purpose of SSL and makes you vulnerable to man-in-the-middle attacks. |
Basic Authentication can
be enabled by adding the following to application.yml
or via
environment variables:
security: basic: enabled: true realm: Spring Cloud Data Flow
Enables basic authentication. Must be set to true for security to be enabled. | |
(Optional) The realm for Basic authentication. Will default to Spring if not explicitly set. |
Note | |
---|---|
Current versions of Chrome do not display the realm. Please see the following Chromium issue ticket for more information. |
In this use-case, the underlying Spring Boot will auto-create a user called user with an auto-generated password which will be printed out to the console upon startup.
Note | |
---|---|
Please be aware of inherent issues of Basic Authentication and logging out, since the credentials are cached by the browser and simply browsing back to application pages will log you back in. |
If you need to define more than one file-based user account, please take a look at File based authentication.
By default Spring Boot allows you to only specify one single user. Spring Cloud Data Flow also supports the listing of more than one user in a configuration file, as described below. Each user must be assigned a password and one or more roles:
security: basic: enabled: true realm: Spring Cloud Data Flow spring: cloud: dataflow: security: authentication: file: enabled: true users: bob: bobspassword, ROLE_MANAGE alice: alicepwd, ROLE_VIEW, ROLE_CREATE
Enables file based authentication | |
This is a yaml map of username to password | |
Each map |
Important | |
---|---|
As of Spring Cloud Data Flow 1.1, roles are not supported, yet (specified roles are ignored). Due to an issue in Spring Security, though, at least one role must be provided. |
Spring Cloud Data Flow also supports authentication against an LDAP server (Lightweight Directory Access Protocol), providing support for the following 2 modes:
When the LDAP authentication option is activated, the default single user mode is turned off.
In direct bind mode, a pattern is defined for the user’s distinguished name (DN), using a placeholder for the username. The authentication process derives the distinguished name of the user by replacing the placeholder and use it to authenticate a user against the LDAP server, along with the supplied password. You can set up LDAP direct bind as follows:
security: basic: enabled: true realm: Spring Cloud Data Flow spring: cloud: dataflow: security: authentication: ldap: enabled: true url: ldap://ldap.example.com:3309 userDnPattern: uid={0},ou=people,dc=example,dc=com
Enables LDAP authentication | |
The URL for the LDAP server | |
The distinguished name (DN) pattern for authenticating against the server |
The search and bind mode involves connecting to an LDAP server, either anonymously or with a fixed account, and searching for the distinguished name of the authenticating user based on its username, and then using the resulting value and the supplied password for binding to the LDAP server. This option is configured as follows:
security: basic: enabled: true realm: Spring Cloud Data Flow spring: cloud: dataflow: security: authentication: ldap: enabled: true url: ldap://localhost:10389 managerDn: uid=admin,ou=system managerPassword: secret userSearchBase: ou=otherpeople,dc=example,dc=com userSearchFilter: uid={0}
Enables LDAP integration | |
The URL of the LDAP server | |
A DN for to authenticate to the LDAP server, if anonymous searches are not supported (optional, required together with next option) | |
A password to authenticate to the LDAP server, if anonymous searches are not supported (optional, required together with previous option) | |
The base for searching the DN of the authenticating user (serves to restrict the scope of the search) | |
The search filter for the DN of the authenticating user |
Tip | |
---|---|
For more information, please also see the chapter LDAP Authentication of the Spring Security reference guide. |
When connecting to an LDAP server, you typically (In the LDAP world) have 2 options in order to establish a connection to an LDAP server securely:
As of Spring Cloud Data Flow 1.1.0 only LDAPs is supported out-of-the-box. When using
official certificates no special configuration is necessary, in order to connect
to an LDAP Server via LDAPs. Just change the url format to ldaps, e.g. ldaps://localhost:636
.
In case of using self-signed certificates, the setup for your Spring Cloud Data Flow server becomes slightly more complex. The setup is very similar to Section 17.1.1, “Using Self-Signed Certificates” (Please read first) and Spring Cloud Data Flow needs to reference a trustStore in order to work with your self-signed certificates.
Important | |
---|---|
While useful during development and testing, please never use self-signed certificates in production! |
Ultimately you have to provide a set of system properties to reference the trustStore and its credentials when starting the server:
$ java -Djavax.net.ssl.trustStorePassword=dataflow \ -Djavax.net.ssl.trustStore=/path/to/dataflow.truststore \ -Djavax.net.ssl.trustStoreType=jks \ -jar spring-cloud-starter-dataflow-server-local-1.2.0.M1.jar
As mentioned above, another option to connect to an LDAP server securely is via Start TLS. In the LDAP world, LDAPs is technically even considered deprecated in favor of Start TLS. However, this option is currently not supported out-of-the-box by Spring Cloud Data Flow.
Please follow the following issue tracker ticket to track its implementation. You may also want to look at the Spring LDAP reference documentation chapter on Custom DirContext Authentication Processing for further details.
All of the above deals with authentication, i.e. how to assess the identity of the user. Irrespective of the option chosen, you can also customize authorization i.e. who can do what.
The default scheme uses three roles to protect the REST endpoints that Spring Cloud Data Flow exposes:
All of those defaults are specified in dataflow-server-defaults.yml
which is
part of the Spring Cloud Data Flow Core Module. Nonetheless, you can
override those, if desired, e.g. in application.yml
. The configuration takes
the form of a YAML list (as some rules may have precedence over others) and so
you’ll need to copy/paste the whole list and tailor it to your needs (as there is no way to merge lists). Always refer to your version of application.yml
, as the snippet reproduced
below may be out-dated. The default rules are as such:
spring: cloud: dataflow: security: authorization: enabled: true rules: # Metrics - GET /metrics/** => hasRole('ROLE_VIEW') - DELETE /metrics/** => hasRole('ROLE_CREATE') # Boot Endpoints - GET /management/** => hasRole('ROLE_MANAGE') # Apps - GET /apps => hasRole('ROLE_VIEW') - GET /apps/** => hasRole('ROLE_VIEW') - DELETE /apps/** => hasRole('ROLE_CREATE') - POST /apps => hasRole('ROLE_CREATE') - POST /apps/** => hasRole('ROLE_CREATE') # Completions - GET /completions/** => hasRole('ROLE_CREATE') # Job Executions & Batch Job Execution Steps && Job Step Execution Progress - GET /jobs/executions => hasRole('ROLE_VIEW') - PUT /jobs/executions/** => hasRole('ROLE_CREATE') - GET /jobs/executions/** => hasRole('ROLE_VIEW') # Batch Job Instances - GET /jobs/instances => hasRole('ROLE_VIEW') - GET /jobs/instances/* => hasRole('ROLE_VIEW') # Running Applications - GET /runtime/apps => hasRole('ROLE_VIEW') - GET /runtime/apps/** => hasRole('ROLE_VIEW') # Stream Definitions - GET /streams/definitions => hasRole('ROLE_VIEW') - GET /streams/definitions/* => hasRole('ROLE_VIEW') - GET /streams/definitions/*/related => hasRole('ROLE_VIEW') - POST /streams/definitions => hasRole('ROLE_CREATE') - DELETE /streams/definitions/* => hasRole('ROLE_CREATE') - DELETE /streams/definitions => hasRole('ROLE_CREATE') # Stream Deployments - DELETE /streams/deployments/* => hasRole('ROLE_CREATE') - DELETE /streams/deployments => hasRole('ROLE_CREATE') - POST /streams/deployments/* => hasRole('ROLE_CREATE') # Task Definitions - POST /tasks/definitions => hasRole('ROLE_CREATE') - DELETE /tasks/definitions/* => hasRole('ROLE_CREATE') - GET /tasks/definitions => hasRole('ROLE_VIEW') - GET /tasks/definitions/* => hasRole('ROLE_VIEW') # Task Executions - GET /tasks/executions => hasRole('ROLE_VIEW') - GET /tasks/executions/* => hasRole('ROLE_VIEW')
The format of each line is the following:
HTTP_METHOD URL_PATTERN '=>' SECURITY_ATTRIBUTE
where
Be mindful that the above is indeed a YAML list, not a map (thus the use of '-' dashes at the start of each line) that lives under the spring.cloud.dataflow.security.authorization.rules
key.
Tip | |
---|---|
In case you are solely interested in authentication but not authorization,
for instance every user shall have have access to all endpoints, then you can also
set |
When configuring Ldap for authentication, you can also specify the group-role-attribute
in conjunction with group-search-base
and group-search-filter
.
The group role attribure contains the name of the role. If not specified, the
ROLE_MANAGE
role is populated by default.
For further information, please refer to Configuring an LDAP Server of the Spring Security reference guide.
OAuth 2.0 allows you to integrate Spring Cloud Data Flow into Single Sign On (SSO) environments. The following 2 OAuth2 Grant Types will be used:
The REST endpoints are secured via Basic Authentication but will use the Password Grand Type under the covers to authenticate with your OAuth2 service.
Note | |
---|---|
When authentication is set up, it is strongly recommended to enable HTTPS as well, especially in production environments. |
You can turn on OAuth2 authentication by adding the following to application.yml
or via
environment variables:
security: basic: enabled: true realm: Spring Cloud Data Flow oauth2: client: client-id: myclient client-secret: mysecret access-token-uri: http://127.0.0.1:9999/oauth/token user-authorization-uri: http://127.0.0.1:9999/oauth/authorize resource: user-info-uri: http://127.0.0.1:9999/me
Must be set to | |
The realm for Basic authentication | |
OAuth Configuration Section, if you leave off the OAuth2 section, Basic Authentication will be enabled instead. |
Note | |
---|---|
As of the current version, Spring Cloud Data Flow does not provide finer-grained authorization when OAUTH is used as authentication mechanism. Thus, once you are logged in, you have full access to all functionality. |
You can verify that basic authentication is working properly using curl:
$ curl -u myusername:mypassword http://localhost:9393/
As a result you should see a list of available REST endpoints.
If your OAuth2 provider supports the Password Grant Type you can start the Data Flow Shell with:
$ java -jar spring-cloud-dataflow-shell-1.2.0.M1.jar \ --dataflow.uri=http://localhost:9393 \ --dataflow.username=my_username --dataflow.password=my_password
Note | |
---|---|
Keep in mind that when authentication for Spring Cloud Data Flow is enabled, the underlying OAuth2 provider must support the Password OAuth2 Grant Type, if you want to use the Shell. |
From within the Data Flow Shell you can also provide credentials using:
dataflow config server --uri http://localhost:9393 --username my_username --password my_password
Once successfully targeted, you should see the following output:
dataflow:>dataflow config info dataflow config info ╔═══════════╤═══════════════════════════════════════╗ ║Credentials│[username='my_username, password=****']║ ╠═══════════╪═══════════════════════════════════════╣ ║Result │ ║ ║Target │http://localhost:9393 ║ ╚═══════════╧═══════════════════════════════════════╝
With Spring Security OAuth you can easily create your own OAuth2 Server with the following 2 simple annotations:
A working example application can be found at:
https://github.com/ghillert/oauth-test-server/
Simply clone the project, built and start it. Furthermore configure Spring Cloud Data Flow with the respective Client Id and Client Secret.
If you rather like to use an existing OAuth2 provider, here is an example for GitHub. First you need to Register a new application under your GitHub account at:
https://github.com/settings/developers
When running a default version of Spring Cloud Data Flow locally, your GitHub configuration should look like the following:
Note | |
---|---|
For the Authorization callback URL you will enter Spring Cloud Data Flow’s Login URL, e.g. |
Configure Spring Cloud Data Flow with the GitHub relevant Client Id and Secret:
security: basic: enabled: true oauth2: client: client-id: your-github-client-id client-secret: your-github-client-secret access-token-uri: https://github.com/login/oauth/access_token user-authorization-uri: https://github.com/login/oauth/authorize resource: user-info-uri: https://api.github.com/user
Important | |
---|---|
GitHub does not support the OAuth2 password grant type. As such you cannot use the Spring Cloud Data Flow Shell in conjunction with GitHub. |
When enabling security, please also make sure that the Spring Boot HTTP Management Endpoints
are secured as well. You can enable security for the management endpoints by adding the following to application.yml
:
management: contextPath: /management security: enabled: true
Important | |
---|---|
If you don’t explicitly enable security for the management endpoints,
you may end up having unsecured REST endpoints, despite |