15. Configuration Reference

The following pieces of configuration must be provided. These are Spring Boot @ConfigurationProperties so you can set them as environment variables or by any other means that Spring Boot supports. Here is a listing in environment variable format as that is an easy way to get started configuring Boot applications in Cloud Foundry.

# Default values cited after the equal sign.
# Example values, typical for Pivotal Web Services, cited as a comment

# url of the CF API (used when using cf login -a for example), e.g. https://api.run.pivotal.io
# (for setting env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL)
spring.cloud.deployer.cloudfoundry.url=

# name of the organization that owns the space above, e.g. youruser-org
# (For Setting Env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG)
spring.cloud.deployer.cloudfoundry.org=

# name of the space into which modules will be deployed, e.g. development
# (for setting env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE)
spring.cloud.deployer.cloudfoundry.space=

# the root domain to use when mapping routes, e.g. cfapps.io
# (for setting env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN)
spring.cloud.deployer.cloudfoundry.domain=

# username and password of the user to use to create apps
# (for setting env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME and SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD)
spring.cloud.deployer.cloudfoundry.username=
spring.cloud.deployer.cloudfoundry.password=

# Whether to allow self-signed certificates during SSL validation
# (for setting env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SKIP_SSL_VALIDATION)
spring.cloud.deployer.cloudfoundry.skipSslValidation=false

# Comma separated set of service instance names to bind to every stream app deployed.
# Amongst other things, this should include a service that will be used
# for Spring Cloud Stream binding, e.g. rabbit
# (for setting env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_SERVICES)
spring.cloud.deployer.cloudfoundry.stream.services=

# Health check type to use for stream apps. Accepts 'none' and 'port'
spring.cloud.deployer.cloudfoundry.stream.health-check=


# Comma separated set of service instance names to bind to every task app deployed.
# Amongst other things, this should include an RDBMS service that will be used
# for Spring Cloud Task execution reporting, e.g. my_mysql
# (for setting env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES)
spring.cloud.deployer.cloudfoundry.task.services=

# Timeout to use, in seconds, when doing blocking API calls to Cloud Foundry.
# (for setting env var use SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_API_TIMEOUT
and SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_API_TIMEOUT)
spring.cloud.deployer.cloudfoundry.stream.apiTimeout=360
spring.cloud.deployer.cloudfoundry.task.apiTimeout=360

Note that you can set the following properties spring.cloud.deployer.cloudfoundry.services, spring.cloud.deployer.cloudfoundry.buildpack or the Spring Cloud Deployer standard spring.cloud.deployer.memory and spring.cloud.deployer.disk as part of an individual deployment request prefixed by the app.<name of application>. For example

>stream create --name ticktock --definition "time | log"
>stream deploy --name ticktock --properties "app.time.spring.cloud.deployer.memory=2g"

will deploy the time source with 2048MB of memory, while the log sink will use the default 1024MB.

15.1 Understanding what’s going on

If you want to get better insights into what is happening when your streams and tasks are being deployed, you may want to turn on the following features:

  • Reactor "stacktraces", showing which operators were involved before an error occurred. This is helpful as the deployer relies on project reactor and regular stacktraces may not always allow understanding the flow before an error happened. Note that this comes with a performance penalty, so is disabled by default.

    spring.cloud.dataflow.server.cloudfoundry.debugReactor = true
  • Deployer and Cloud Foundry client library request/response logs. This allows seeing detailed conversation between the Data Flow server and the Cloud Foundry Cloud Controller.

    logging.level.cloudfoundry-client = DEBUG

15.2 Using Spring Cloud Config Server

Spring Cloud Config Server can be used to centralize configuration properties for Spring Boot applications. Likewise, both Spring Cloud Data Flow and the applications orchestrated using Spring Cloud Data Flow can be integrated with config-server to leverage the same capabilities. Let’s review the integration steps for both the variants.

15.2.1 Spring Cloud Data Flow and Spring Cloud Config Server

  • Download 1.4.x release of Spring Boot from start.spring.io
  • Add spring-cloud-dataflow-server-cloudfoundry-autoconfig dependency pointing to Spring Cloud Data Flow’s Cloud Foundry release
  • Add @EnableDataFlowServer to the downloaded application
@SpringBootApplication
@EnableDataFlowServer
public class DataFlowServer {

	public static void main(String[] args) {
		new SpringApplication(DataFlowServer.class).run(args);
	}
}
  • Add spring-cloud-services-starter-config-client dependency in pom.xml. A maven example follows.
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-dataflow-server-cloudfoundry-autoconfig</artifactId>
  <version>CF_SERVER_VERSION</version>
</dependency>
<dependency>
  <groupId>io.pivotal.spring.cloud</groupId>
  <artifactId>spring-cloud-services-starter-config-client</artifactId>
  <version>CONFIG_CLIENT_VERSION</version>
</dependency>

Where, CF_SERVER_VERSION and CONFIG_CLIENT_VERSION tokens can be the latest release versions of SCDF’s CF-server and Spring Cloud Config Server client for Pivotal Cloud Foundry respectively.

  • Build the application locally

This completes the custom build of Spring Cloud Data Flow with spring-cloud-services-starter-config-client library included in it as a dependency.

The final über-jar is now ready to be deployed to Cloud Foundry. With this setup and having the deployed application bound to config-server service on Cloud Foundry, we can successfully negotiate, read, and resolve centralized properties at the runtime.

Follow the documentation for Config Server for Pivotal Cloud Foundry. For more details, please refer to Spring Cloud Services client-dependencies documentation.

15.2.2 Stream, Task, and Spring Cloud Config Server

Similar to Spring Cloud Data Flow server, it is also possible to configure both the stream and task applications to resolve the centralized properties from config-server.

Let’s assume you’d like to read properties for time-source application from config-server.

  • Download the time-source application starter with "Rabbit binder starter" from start-scs.cfapps.io/
  • Load the downloaded project in an IDE
  • Add @Import(org.springframework.cloud.stream.app.time.source.TimeSourceConfiguration.class) to import time-source’s configuration properties
  • Add spring-cloud-services-starter-config-client dependency
  • Build the application locally

This completes the custom build of time-source application with spring-cloud-services-starter-config-client library included in it as a dependency. A maven example follows.

<dependency>
  <groupId>org.springframework.cloud.stream.app</groupId>
  <artifactId>spring-cloud-starter-stream-source-time</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-stream-rabbit</artifactId>
</dependency>
<dependency>
  <groupId>io.pivotal.spring.cloud</groupId>
  <artifactId>spring-cloud-services-starter-config-client</artifactId>
  <version>CONFIG_CLIENT_VERSION</version>
</dependency>

Where, CONFIG_CLIENT_VERSION can be the latest release of Spring Cloud Config Server client for Pivotal Cloud Foundry.

The final time-source über-jar is now ready to be registered in Spring Cloud Data Flow. For more details, review how to register applications. With this setup and having the deployed application bound to config-server service on Cloud Foundry, we can successfully negotiate, read, and resolve centralized properties at the runtime.

[Note]Note

When deploying apps using Data Flow for Cloud Foundry, a typical choice is to use maven:// coordinates, or maybe http:// URIs.

15.2.3 Sample Manifest Template

Following manifest.yml template includes the required env-var’s for the Spring Cloud Data Flow server to successfully run on Cloud Foundry and automatically resolve centralized properties from config-server at the runtime.

---
applications:
- name: test-server
  host: test-server
  memory: 1G
  disk_quota: 1G
  instances: 1
  path: spring-cloud-dataflow-server-cloudfoundry-VERSION.jar
  env:
    SPRING_APPLICATION_NAME: test-server
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL: <URL>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG: <ORG>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE: <SPACE>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN: <DOMAIN>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME: <USER>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD: <PASSWORD>
    MAVEN_REMOTE_REPOSITORIES_REPO1_URL: https://repo.spring.io/libs-release
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_SERVICES: config-server #this is for all the stream applications
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES: config-server #this is for all the task applications
services:
- mysql
- config-server #this is for the server

Where, config-server is the name of the Spring Cloud Config Service instance running on Cloud Foundry. By binding the service to both Spring Cloud Data Flow server and as well as all the Spring Cloud Stream and Spring Cloud Task applications through SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_SERVICES: config-server and SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES: config-server respectively, we can now resolve centralized properties backed by this service.

15.2.4 Self-signed SSL Certificate and Spring Cloud Config Server

Often, in a development environment, we may not have a valid certificate to enable SSL communication between clients and the backend services. However, the config-server for Pivotal Cloud Foundry uses HTTPS for all client-to-service communication, so it is necessary to add a self-signed SSL certificate in environments with no valid certificates.

Using the same manifest.yml template listed in the previous section, for the server, we can provide the self-signed SSL certificate via: TRUST_CERTS: <API_ENDPOINT>.

For Spring Cloud Stream and Spring Cloud Task applications, it is necessary to wrap the TRUST_CERTS in SPRING_APPLICATION_JSON token - this instructs the server to propagate SPRING_APPLICATION_JSON content to all the deployed applications.

However, the deployed applications require TRUST_CERTS as a flat env-var as opposed to being wrapped inside SPRING_APPLICATION_JSON, so we will have to instruct the server with yet another set of tokens SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_USE_SPRING_APPLICATION_JSON: false and SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_USE_SPRING_APPLICATION_JSON: false for stream and task applications respectively. With this setup, the applications will be deployed with the content in SPRING_APPLICATION_JSON as flat env-var’s.

Let’s review the updated manifest.yml with the required changes.

---
applications:
- name: test-server
  host: test-server
  memory: 1G
  disk_quota: 1G
  instances: 1
  path: spring-cloud-dataflow-server-cloudfoundry-VERSION.jar
  env:
    SPRING_APPLICATION_NAME: test-server
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_URL: <URL>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_ORG: <ORG>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_SPACE: <SPACE>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_DOMAIN: <DOMAIN>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME: <USER>
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD: <PASSWORD>
    MAVEN_REMOTE_REPOSITORIES_REPO1_URL: https://repo.spring.io/libs-release
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_SERVICES: config-server #this is for all the stream applications
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_SERVICES: config-server #this is for all the task applications
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_STREAM_USE_SPRING_APPLICATION_JSON: false #this is for all the stream applications
    SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_TASK_USE_SPRING_APPLICATION_JSON: false #this is for all the task applications
    TRUST_CERTS: <API_ENDPOINT> #this is for the server
    SPRING_APPLICATION_JSON: '{"spring.cloud.dataflow.applicationProperties.stream.TRUST_CERTS" : <API_ENDPOINT>,"spring.cloud.dataflow.applicationProperties.task.TRUST_CERTS" : <API_ENDPOINT>}'
services:
- mysql
- config-server #this is for the server