17. Security

By default, the Data Flow server is unsecured and runs on an unencrypted HTTP connection. You can secure your REST endpoints, as well as the Data Flow Dashboard by enabling HTTPS and requiring clients to authenticate using either:

NOTE: By default, the REST endpoints (administration, management and health), as well as the Dashboard UI do not require authenticated access.

17.1 Enabling HTTPS

By default, the dashboard, management, and health endpoints use HTTP as a transport. You can switch to HTTPS easily, by adding a certificate to your configuration in application.yml.

server:
  port: 8443                                         1
  ssl:
    key-alias: yourKeyAlias                          2
    key-store: path/to/keystore                      3
    key-store-password: yourKeyStorePassword         4
    key-password: yourKeyPassword                    5
    trust-store: path/to/trust-store                 6
    trust-store-password: yourTrustStorePassword     7

1

As the default port is 9393, you may choose to change the port to a more common HTTPs-typical port.

2

The alias (or name) under which the key is stored in the keystore.

3

The path to the keystore file. Classpath resources may also be specified, by using the classpath prefix: classpath:path/to/keystore

4

The password of the keystore.

5

The password of the key.

6

The path to the truststore file. Classpath resources may also be specified, by using the classpath prefix: classpath:path/to/trust-store

7

The password of the trust store.

[Note]Note

If HTTPS is enabled, it will completely replace HTTP as the protocol over which the REST endpoints and the Data Flow Dashboard interact. Plain HTTP requests will fail - therefore, make sure that you configure your Shell accordingly.

17.1.1 Using Self-Signed Certificates

For testing purposes or during development it might be convenient to create self-signed certificates. To get started, execute the following command to create a certificate:

$ keytool -genkey -alias dataflow -keyalg RSA -keystore dataflow.keystore \
          -validity 3650 -storetype JKS \
          -dname "CN=localhost, OU=Spring, O=Pivotal, L=Kailua-Kona, ST=HI, C=US"  1
          -keypass dataflow -storepass dataflow

1

CN is the only important parameter here. It should match the domain you are trying to access, e.g. localhost.

Then add the following to your application.yml file:

server:
  port: 8443
  ssl:
    enabled: true
    key-alias: dataflow
    key-store: "/your/path/to/dataflow.keystore"
    key-store-type: jks
    key-store-password: dataflow
    key-password: dataflow

This is all that’s needed for the Data Flow Server. Once you start the server, you should be able to access it via https://localhost:8443/. As this is a self-signed certificate, you will hit a warning in your browser, that you need to ignore.

17.1.2 Self-Signed Certificates and the Shell

By default self-signed certificates are an issue for the Shell and additional steps are necessary to make the Shell work with self-signed certificates. Two options are available:

  1. Add the self-signed certificate to the JVM truststore
  2. Skip certificate validation

Add the self-signed certificate to the JVM truststore

In order to use the JVM truststore option, we need to export the previously created certificate from the keystore:

$ keytool -export -alias dataflow -keystore dataflow.keystore -file dataflow_cert -storepass dataflow

Next, we need to create a truststore which the Shell will use:

$ keytool -importcert -keystore dataflow.truststore -alias dataflow -storepass dataflow -file dataflow_cert -noprompt

Now, you are ready to launch the Data Flow Shell using the following JVM arguments:

$ java -Djavax.net.ssl.trustStorePassword=dataflow \
       -Djavax.net.ssl.trustStore=/path/to/dataflow.truststore \
       -Djavax.net.ssl.trustStoreType=jks \
       -jar spring-cloud-dataflow-shell-1.2.0.M1.jar
[Tip]Tip

In case you run into trouble establishing a connection via SSL, you can enable additional logging by using and setting the javax.net.debug JVM argument to ssl.

Don’t forget to target the Data Flow Server with:

dataflow:> dataflow config server https://localhost:8443/

Skip Certificate Validation

Alternatively, you can also bypass the certification validation by providing the optional command-line parameter --dataflow.skip-ssl-validation=true.

Using this command-line parameter, the shell will accept any (self-signed) SSL certificate.

[Warning]Warning

If possible you should avoid using this option. Disabling the trust manager defeats the purpose of SSL and makes you vulnerable to man-in-the-middle attacks.

17.2 Basic Authentication

Basic Authentication can be enabled by adding the following to application.yml or via environment variables:

security:
  basic:
    enabled: true                                                     1
    realm: Spring Cloud Data Flow                                     2

1

Enables basic authentication. Must be set to true for security to be enabled.

2

(Optional) The realm for Basic authentication. Will default to Spring if not explicitly set.

[Note]Note

Current versions of Chrome do not display the realm. Please see the following Chromium issue ticket for more information.

In this use-case, the underlying Spring Boot will auto-create a user called user with an auto-generated password which will be printed out to the console upon startup.

Figure 17.1. Default Spring Boot user credentials

Default Spring Boot user credentials

[Note]Note

Please be aware of inherent issues of Basic Authentication and logging out, since the credentials are cached by the browser and simply browsing back to application pages will log you back in.

If you need to define more than one file-based user account, please take a look at File based authentication.

17.2.1 File based authentication

By default Spring Boot allows you to only specify one single user. Spring Cloud Data Flow also supports the listing of more than one user in a configuration file, as described below. Each user must be assigned a password and one or more roles:

security:
  basic:
    enabled: true
    realm: Spring Cloud Data Flow
spring:
  cloud:
    dataflow:
      security:
        authentication:
          file:
            enabled: true                                                 1
            users:                                                        2
              bob: bobspassword, ROLE_MANAGE                              3
              alice: alicepwd, ROLE_VIEW, ROLE_CREATE

1

Enables file based authentication

2

This is a yaml map of username to password

3

Each map value is made of a corresponding password and role(s), comma separated

[Important]Important

As of Spring Cloud Data Flow 1.1, roles are not supported, yet (specified roles are ignored). Due to an issue in Spring Security, though, at least one role must be provided.

17.2.2 LDAP Authentication

Spring Cloud Data Flow also supports authentication against an LDAP server (Lightweight Directory Access Protocol), providing support for the following 2 modes:

  • Direct bind
  • Search and bind

When the LDAP authentication option is activated, the default single user mode is turned off.

In direct bind mode, a pattern is defined for the user’s distinguished name (DN), using a placeholder for the username. The authentication process derives the distinguished name of the user by replacing the placeholder and use it to authenticate a user against the LDAP server, along with the supplied password. You can set up LDAP direct bind as follows:

security:
  basic:
    enabled: true
    realm: Spring Cloud Data Flow
spring:
  cloud:
    dataflow:
      security:
        authentication:
          ldap:
            enabled: true                                                 1
            url: ldap://ldap.example.com:3309                             2
            userDnPattern: uid={0},ou=people,dc=example,dc=com            3

1

Enables LDAP authentication

2

The URL for the LDAP server

3

The distinguished name (DN) pattern for authenticating against the server

The search and bind mode involves connecting to an LDAP server, either anonymously or with a fixed account, and searching for the distinguished name of the authenticating user based on its username, and then using the resulting value and the supplied password for binding to the LDAP server. This option is configured as follows:

security:
  basic:
    enabled: true
    realm: Spring Cloud Data Flow
spring:
  cloud:
    dataflow:
      security:
        authentication:
          ldap:
            enabled: true                                                 1
            url: ldap://localhost:10389                                   2
            managerDn: uid=admin,ou=system                                3
            managerPassword: secret                                       4
            userSearchBase: ou=otherpeople,dc=example,dc=com              5
            userSearchFilter: uid={0}                                     6

1

Enables LDAP integration

2

The URL of the LDAP server

3

A DN for to authenticate to the LDAP server, if anonymous searches are not supported (optional, required together with next option)

4

A password to authenticate to the LDAP server, if anonymous searches are not supported (optional, required together with previous option)

5

The base for searching the DN of the authenticating user (serves to restrict the scope of the search)

6

The search filter for the DN of the authenticating user

[Tip]Tip

For more information, please also see the chapter LDAP Authentication of the Spring Security reference guide.

LDAP Transport Security

When connecting to an LDAP server, you typically (In the LDAP world) have 2 options in order to establish a connection to an LDAP server securely:

  • LDAP over SSL (LDAPs)
  • Start Transport Layer Security (Start TLS is defined in RFC2830)

As of Spring Cloud Data Flow 1.1.0 only LDAPs is supported out-of-the-box. When using official certificates no special configuration is necessary, in order to connect to an LDAP Server via LDAPs. Just change the url format to ldaps, e.g. ldaps://localhost:636.

In case of using self-signed certificates, the setup for your Spring Cloud Data Flow server becomes slightly more complex. The setup is very similar to Section 17.1.1, “Using Self-Signed Certificates” (Please read first) and Spring Cloud Data Flow needs to reference a trustStore in order to work with your self-signed certificates.

[Important]Important

While useful during development and testing, please never use self-signed certificates in production!

Ultimately you have to provide a set of system properties to reference the trustStore and its credentials when starting the server:

$ java -Djavax.net.ssl.trustStorePassword=dataflow \
       -Djavax.net.ssl.trustStore=/path/to/dataflow.truststore \
       -Djavax.net.ssl.trustStoreType=jks \
       -jar spring-cloud-starter-dataflow-server-local-1.2.0.M1.jar

As mentioned above, another option to connect to an LDAP server securely is via Start TLS. In the LDAP world, LDAPs is technically even considered deprecated in favor of Start TLS. However, this option is currently not supported out-of-the-box by Spring Cloud Data Flow.

Please follow the following issue tracker ticket to track its implementation. You may also want to look at the Spring LDAP reference documentation chapter on Custom DirContext Authentication Processing for further details.

17.2.3 Customizing authorization

All of the above deals with authentication, i.e. how to assess the identity of the user. Irrespective of the option chosen, you can also customize authorization i.e. who can do what.

The default scheme uses three roles to protect the REST endpoints that Spring Cloud Data Flow exposes:

  • ROLE_VIEW for anything that relates to retrieving state
  • ROLE_CREATE for anything that involves creating, deleting or mutating the state of the system
  • ROLE_MANAGE for boot management endpoints.

All of those defaults are specified in dataflow-server-defaults.yml which is part of the Spring Cloud Data Flow Core Module. Nonetheless, you can override those, if desired, e.g. in application.yml. The configuration takes the form of a YAML list (as some rules may have precedence over others) and so you’ll need to copy/paste the whole list and tailor it to your needs (as there is no way to merge lists). Always refer to your version of application.yml, as the snippet reproduced below may be out-dated. The default rules are as such:

spring:
  cloud:
    dataflow:
      security:
        authorization:
          enabled: true
          rules:
            # Metrics

            - GET    /metrics/**                     => hasRole('ROLE_VIEW')
            - DELETE /metrics/**                     => hasRole('ROLE_CREATE')

            # Boot Endpoints

            - GET    /management/**                  => hasRole('ROLE_MANAGE')

            # Apps

            - GET    /apps                           => hasRole('ROLE_VIEW')
            - GET    /apps/**                        => hasRole('ROLE_VIEW')
            - DELETE /apps/**                        => hasRole('ROLE_CREATE')
            - POST   /apps                           => hasRole('ROLE_CREATE')
            - POST   /apps/**                        => hasRole('ROLE_CREATE')

            # Completions

            - GET /completions/**                    => hasRole('ROLE_CREATE')

            # Job Executions & Batch Job Execution Steps && Job Step Execution Progress

            - GET    /jobs/executions                => hasRole('ROLE_VIEW')
            - PUT    /jobs/executions/**             => hasRole('ROLE_CREATE')
            - GET    /jobs/executions/**             => hasRole('ROLE_VIEW')

            # Batch Job Instances

            - GET    /jobs/instances                 => hasRole('ROLE_VIEW')
            - GET    /jobs/instances/*               => hasRole('ROLE_VIEW')

            # Running Applications

            - GET    /runtime/apps                   => hasRole('ROLE_VIEW')
            - GET    /runtime/apps/**                => hasRole('ROLE_VIEW')

            # Stream Definitions

            - GET    /streams/definitions            => hasRole('ROLE_VIEW')
            - GET    /streams/definitions/*          => hasRole('ROLE_VIEW')
            - GET    /streams/definitions/*/related  => hasRole('ROLE_VIEW')
            - POST   /streams/definitions            => hasRole('ROLE_CREATE')
            - DELETE /streams/definitions/*          => hasRole('ROLE_CREATE')
            - DELETE /streams/definitions            => hasRole('ROLE_CREATE')

            # Stream Deployments

            - DELETE /streams/deployments/*          => hasRole('ROLE_CREATE')
            - DELETE /streams/deployments            => hasRole('ROLE_CREATE')
            - POST   /streams/deployments/*          => hasRole('ROLE_CREATE')

            # Task Definitions

            - POST   /tasks/definitions              => hasRole('ROLE_CREATE')
            - DELETE /tasks/definitions/*            => hasRole('ROLE_CREATE')
            - GET    /tasks/definitions              => hasRole('ROLE_VIEW')
            - GET    /tasks/definitions/*            => hasRole('ROLE_VIEW')

            # Task Executions

            - GET    /tasks/executions               => hasRole('ROLE_VIEW')
            - GET    /tasks/executions/*             => hasRole('ROLE_VIEW')

The format of each line is the following:

HTTP_METHOD URL_PATTERN '=>' SECURITY_ATTRIBUTE

where

Be mindful that the above is indeed a YAML list, not a map (thus the use of '-' dashes at the start of each line) that lives under the spring.cloud.dataflow.security.authorization.rules key.

[Tip]Tip

In case you are solely interested in authentication but not authorization, for instance every user shall have have access to all endpoints, then you can also set spring.cloud.dataflow.security.authorization.enabled=false.

17.2.4 Authorization with Ldap

When configuring Ldap for authentication, you can also specify the group-role-attribute in conjunction with group-search-base and group-search-filter.

The group role attribure contains the name of the role. If not specified, the ROLE_MANAGE role is populated by default.

For further information, please refer to Configuring an LDAP Server of the Spring Security reference guide.

17.3 OAuth 2.0

OAuth 2.0 allows you to integrate Spring Cloud Data Flow into Single Sign On (SSO) environments. The following 2 OAuth2 Grant Types will be used:

  • Authorization Code - Used for the GUI (Browser) integration. You will be redirected to your OAuth Service for authentication
  • Password - Used by the shell (And the REST integration), so you can login using username and password

The REST endpoints are secured via Basic Authentication but will use the Password Grand Type under the covers to authenticate with your OAuth2 service.

[Note]Note

When authentication is set up, it is strongly recommended to enable HTTPS as well, especially in production environments.

You can turn on OAuth2 authentication by adding the following to application.yml or via environment variables:

security:
  basic:
    enabled: true                                                     1
    realm: Spring Cloud Data Flow                                     2
  oauth2:                                                             3
    client:
      client-id: myclient
      client-secret: mysecret
      access-token-uri: http://127.0.0.1:9999/oauth/token
      user-authorization-uri: http://127.0.0.1:9999/oauth/authorize
    resource:
      user-info-uri: http://127.0.0.1:9999/me

1

Must be set to true for security to be enabled.

2

The realm for Basic authentication

3

OAuth Configuration Section, if you leave off the OAuth2 section, Basic Authentication will be enabled instead.

[Note]Note

As of the current version, Spring Cloud Data Flow does not provide finer-grained authorization when OAUTH is used as authentication mechanism. Thus, once you are logged in, you have full access to all functionality.

You can verify that basic authentication is working properly using curl:

$ curl -u myusername:mypassword http://localhost:9393/

As a result you should see a list of available REST endpoints.

17.3.1 Authentication using the Spring Cloud Data Flow Shell

If your OAuth2 provider supports the Password Grant Type you can start the Data Flow Shell with:

$ java -jar spring-cloud-dataflow-shell-1.2.0.M1.jar \
  --dataflow.uri=http://localhost:9393 \
  --dataflow.username=my_username --dataflow.password=my_password
[Note]Note

Keep in mind that when authentication for Spring Cloud Data Flow is enabled, the underlying OAuth2 provider must support the Password OAuth2 Grant Type, if you want to use the Shell.

From within the Data Flow Shell you can also provide credentials using:

dataflow config server --uri http://localhost:9393 --username my_username --password my_password

Once successfully targeted, you should see the following output:

dataflow:>dataflow config info
dataflow config info

╔═══════════╤═══════════════════════════════════════╗
║Credentials│[username='my_username, password=****']║
╠═══════════╪═══════════════════════════════════════╣
║Result     │                                       ║
║Target     │http://localhost:9393                  ║
╚═══════════╧═══════════════════════════════════════╝

17.3.2 OAuth2 Authentication Examples

Local OAuth2 Server

With Spring Security OAuth you can easily create your own OAuth2 Server with the following 2 simple annotations:

  • @EnableResourceServer
  • @EnableAuthorizationServer

A working example application can be found at:

https://github.com/ghillert/oauth-test-server/

Simply clone the project, built and start it. Furthermore configure Spring Cloud Data Flow with the respective Client Id and Client Secret.

Authentication using GitHub

If you rather like to use an existing OAuth2 provider, here is an example for GitHub. First you need to Register a new application under your GitHub account at:

https://github.com/settings/developers

When running a default version of Spring Cloud Data Flow locally, your GitHub configuration should look like the following:

Figure 17.2. Register an OAuth Application for GitHub

Register an OAuth Application for GitHub

[Note]Note

For the Authorization callback URL you will enter Spring Cloud Data Flow’s Login URL, e.g. localhost:9393/login.

Configure Spring Cloud Data Flow with the GitHub relevant Client Id and Secret:

security:
  basic:
    enabled: true
  oauth2:
    client:
      client-id: your-github-client-id
      client-secret: your-github-client-secret
      access-token-uri: https://github.com/login/oauth/access_token
      user-authorization-uri: https://github.com/login/oauth/authorize
    resource:
      user-info-uri: https://api.github.com/user
[Important]Important

GitHub does not support the OAuth2 password grant type. As such you cannot use the Spring Cloud Data Flow Shell in conjunction with GitHub.

17.4 Securing the Spring Boot Management Endpoints

When enabling security, please also make sure that the Spring Boot HTTP Management Endpoints are secured as well. You can enable security for the management endpoints by adding the following to application.yml:

management:
  contextPath: /management
  security:
    enabled: true
[Important]Important

If you don’t explicitly enable security for the management endpoints, you may end up having unsecured REST endpoints, despite security.basic.enabled being set to true.