Spring Cloud Kubernetes

Kubernetes provides a resource named ConfigMap to externalize the parameters to pass to your application in the form of key-value pairs or embedded application.properties or application.yaml files. The Spring Cloud Kubernetes Config project makes Kubernetes ConfigMap instances available during application startup and triggers hot reloading of beans or Spring context when changes are detected on observed ConfigMap instances.

Everything that follows is explained mainly referring to examples using ConfigMaps, but the same stands for Secrets, i.e.: every feature is supported for both.

The default behavior is to create a Fabric8ConfigMapPropertySource (or a KubernetesClientConfigMapPropertySource) based on a Kubernetes ConfigMap that has metadata.name of either:

However, more advanced configuration is possible where you can use multiple ConfigMap instances. The spring.cloud.kubernetes.config.sources list makes this possible. For example, you could define the following ConfigMap instances:

In the preceding example, if spring.cloud.kubernetes.config.namespace had not been set, the ConfigMap named c1 would be looked up in the namespace that the application runs. See Namespace resolution to get a better understanding of how the namespace of the application is resolved.

Any matching ConfigMap that is found is processed as follows:

An example should make a lot more sense. Let’s suppose that spring.application.name=my-app and that we have a single active profile called k8s. For a configuration as below:

This is what we will end-up loading:

The order of loading properties is a as follows:

This means that profile based sources take precedence over non-profile based sources (just like in a vanilla Spring app); and plain properties take precedence over both profile and non-profile based sources. Here is an example:

After processing such a ConfigMap, this is what you will get in the properties: key1=valueD, key2=valueB.

The single exception to the aforementioned flow is when the ConfigMap contains a single key that indicates the file is a YAML or properties file. In that case, the name of the key does NOT have to be application.yaml or application.properties (it can be anything) and the value of the property is treated correctly. This features facilitates the use case where the ConfigMap was created by using something like the following:

Assume that we have a Spring Boot application named demo that uses the following properties to read its thread pool configuration.

This can be externalized to config map in yaml format as follows:

Individual properties work fine for most cases. However, sometimes, embedded yaml is more convenient. In this case, we use a single property named application.yaml to embed our yaml, as follows:

The following example also works:

You can also define the search to happen based on labels, for example:

This will search for every configmap in namespace spring-k8s that has labels {letter : a}. The important thing to notice here is that unlike reading a configmap by name, this can result in multiple config maps read. As usual, the same feature is supported for secrets.

You can also configure Spring Boot applications differently depending on active profiles that are merged together when the ConfigMap is read. You can provide different property values for different profiles by using an application.properties or application.yaml property, specifying profile-specific values, each in their own document (indicated by the --- sequence), as follows:

In the preceding case, the configuration loaded into your Spring Application with the development profile is as follows:

However, if the production profile is active, the configuration becomes:

If both profiles are active, the property that appears last within the ConfigMap overwrites any preceding values.

Another option is to create a different config map per profile and spring boot will automatically fetch it based on active profiles

To tell Spring Boot which profile should be enabled see the Spring Boot documentation. One option for activating a specific profile when deploying to Kubernetes is to launch your Spring Boot application with an environment variable that you can define in the PodSpec at the container specification. Deployment resource file, as follows:

You could run into a situation where there are multiple configs maps that have the same property names. For example:

and

Depending on the order in which you place these in bootstrap.yaml|properties, you might end up with an un-expected result (the last config map wins). For example:

will result in property greetings.message being Say Hello from one.

There is a way to change this default configuration by specifying useNameAsPrefix. For example:

Such a configuration will result in two properties being generated:

Notice that spring.cloud.kubernetes.config.useNameAsPrefix has a lower priority than spring.cloud.kubernetes.config.sources.useNameAsPrefix. This allows you to set a "default" strategy for all sources, at the same time allowing to override only a few.

If using the config map name is not an option, you can specify a different strategy, called : explicitPrefix. Since this is an explicit prefix that you select, it can only be supplied to the sources level. At the same time it has a higher priority than useNameAsPrefix. Let’s suppose we have a third config map with these entries:

A configuration like the one below:

will result in three properties being generated:

The same way you configure a prefix for configmaps, you can do it for secrets also; both for secrets that are based on name and the ones based on labels. For example:

The same processing rules apply when generating property source as for config maps. The only difference is that potentially, looking up secrets by labels can mean that we find more than one source. In such a case, prefix (if specified via useNameAsPrefix) will be the names of all secrets found for those particular labels.

One more thing to bear in mind is that we support prefix per source, not per secret. The easiest way to explain this is via an example:

Suppose that a query matching such a label will provide two secrets as a result: secret-a and secret-b. Both of these secrets have the same property name: color=sea-blue and color=ocean-blue. It is undefined which color will end-up as part of property sources, but the prefix for it will be secret-a.secret-b (concatenated sorted naturally, names of the secrets).

If you need more fine-grained results, adding more labels to identify the secret uniquely would be an option.

By default, besides reading the config map that is specified in the sources configuration, Spring will also try to read all properties from "profile aware" sources. The easiest way to explain this is via an example. Let’s suppose your application enables a profile called "dev" and you have a configuration like the one below:

Besides reading the config-map-one, Spring will also try to read config-map-one-dev; in this particular order. Each active profile generates such a profile aware config map.

Though your application should not be impacted by such a config map, it can be disabled if needed:

Notice that just like before, there are two levels where you can specify this property: for all config maps or for individual ones; the latter having a higher priority.

Another option for using ConfigMap instances is to mount them into the Pod by running the Spring Cloud Kubernetes application and having Spring Cloud Kubernetes read them from the file system.

In some cases, your application may be unable to load some of your ConfigMaps using the Kubernetes API. If you want your application to fail the start-up process in such cases, you can set spring.cloud.kubernetes.config.fail-fast=true to make the application start-up fail with an Exception.

You can also make your application retry loading ConfigMap property sources on a failure. First, you need to set spring.cloud.kubernetes.config.fail-fast=true. Then you need to add spring-retry and spring-boot-starter-aop to your classpath. You can configure retry properties such as the maximum number of attempts, backoff options like initial interval, multiplier, max interval by setting the spring.cloud.kubernetes.config.retry.* properties.

Kubernetes has the notion of Secrets for storing sensitive data such as passwords, OAuth tokens, and so on. This project provides integration with Secrets to make secrets accessible by Spring Boot applications. You can explicitly enable or disable This feature by setting the spring.cloud.kubernetes.secrets.enabled property.

When enabled, the Fabric8SecretsPropertySource looks up Kubernetes for Secrets from the following sources:

Note:

By default, consuming Secrets through the API (points 2 and 3 above) is not enabled for security reasons. The permission 'list' on secrets allows clients to inspect secrets values in the specified namespace. Further, we recommend that containers share secrets through mounted volumes.

If you enable consuming Secrets through the API, we recommend that you limit access to Secrets by using an authorization policy, such as RBAC. For more information about risks and best practices when consuming Secrets through the API refer to this doc.

If the secrets are found, their data is made available to the application.

Assume that we have a spring boot application named demo that uses properties to read its database configuration. We can create a Kubernetes secret by using the following command:

The preceding command would create the following secret (which you can see by using kubectl get secrets db-secret -o yaml):

Note that the data contains Base64-encoded versions of the literal provided by the create command.

Your application can then use this secret — for example, by exporting the secret’s value as environment variables:

You can select the Secrets to consume in a number of ways:

As the case with ConfigMap, more advanced configuration is also possible where you can use multiple Secret instances. The spring.cloud.kubernetes.secrets.sources list makes this possible. For example, you could define the following Secret instances:

In the preceding example, if spring.cloud.kubernetes.secrets.namespace had not been set, the Secret named s1 would be looked up in the namespace that the application runs. See namespace-resolution to get a better understanding of how the namespace of the application is resolved.

Similar to the ConfigMaps; if you want your application to fail to start when it is unable to load Secrets property sources, you can set spring.cloud.kubernetes.secrets.fail-fast=true.

It is also possible to enable retry for Secret property sources like the ConfigMaps. As with the ConfigMap property sources, first you need to set spring.cloud.kubernetes.secrets.fail-fast=true. Then you need to add spring-retry and spring-boot-starter-aop to your classpath. Retry behavior of the Secret property sources can be configured by setting the spring.cloud.kubernetes.secrets.retry.* properties.

Since data coming from Secrets is usually treated as sensitive, endpoints of the actuator /env and /configprops can be made to sanitize data, so that it is not displayed in plain text. In order to do that, you need to set:

This setting is supported since 3.0.6 and upwards.

Notes:

You can find an example of an application that uses secrets (though it has not been updated to use the new spring-cloud-kubernetes project) at spring-boot-camel-config

Finding an application namespace happens on a best-effort basis. There are some steps that we iterate in order to find it. The easiest and most common one, is to specify it in the proper configuration, for example:

Remember that the same can be done for config maps. If such a namespace is not specified, it will be read (in this order):

Failure to find a namespace from the above steps will result in an Exception being raised.

If, for whatever reason, you enabled both configmaps and secrets, and there is a common property between them, the value from the ConfigMap will have a higher precedence. That is: it will override whatever values are found in secrets.

Some applications may need to detect changes on external property sources and update their internal status to reflect the new configuration. The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related ConfigMap or Secret changes.

By default, this feature is disabled. You can enable it by using the spring.cloud.kubernetes.reload.enabled=true configuration property (for example, in the application.properties file). Please notice that this will enable monitoring of configmaps only (i.e.: spring.cloud.kubernetes.reload.monitoring-config-maps will be set to true). If you want to enable monitoring of secrets, this must be done explicitly via : spring.cloud.kubernetes.reload.monitoring-secrets=true.

The following levels of reload are supported (by setting the spring.cloud.kubernetes.reload.strategy property):

Assuming that the reload feature is enabled with default settings (refresh mode), the following bean is refreshed when the config map changes:

To see that changes effectively happen, you can create another bean that prints the message periodically, as follows

You can change the message printed by the application by using a ConfigMap, as follows:

Any change to the property named bean.message in the ConfigMap associated with the pod is reflected in the output. More generally speaking, changes associated to properties prefixed with the value defined by the prefix field of the @ConfigurationProperties annotation are detected and reflected in the application. Associating a ConfigMap with a pod is explained earlier in this chapter.

The reload feature supports two operating modes:

By default, a namespace chosen using the steps outlined in Namespace resolution will be used to listen to changes in configmaps and secrets. i.e.: if you do not tell reload what namespaces and configmaps/secrets to watch for, it will watch all configmaps/secrets from the namespace that will be computed using the above algorithm.

On the other hand, you can define a more fine-grained approach. For example, you can specify the namespaces where changes will be monitored:

Such a configuration will make the app watch changes only in the my-namespace namespace. Mind that this will watch all configmaps/secrets (depending on which one you enable). If you want an even more fine-grained approach, you can enable "label-filtering". First we need to enable such support via : enable-reload-filtering: true

What this will do, is watch configmaps/secrets that only have the spring.cloud.kubernetes.config.informer.enabled: true label.

Notes:

In versions of Spring Cloud Kubernetes prior to 3.0.x, Kubernetes awareness was implemented using spring.cloud.kubernetes.enabled property. This property was removed and is un-supported. Instead, we use Spring Boot API: ConditionalOnCloudPlatform. If it is needed to explicitly enable or disable this awareness, use spring.main.cloud-platform=NONE/KUBERNETES.

When the application runs as a pod inside Kubernetes, a Spring profile named kubernetes automatically gets activated. This lets you customize the configuration, to define beans that are applied when the Spring Boot application is deployed within the Kubernetes platform (for example, different development and production configuration).

When you include the spring-cloud-kubernetes-fabric8-istio module in the application classpath, a new profile is added to the application, provided the application is running inside a Kubernetes Cluster with Istio installed. You can then use spring @Profile("istio") annotations in your Beans and @Configuration classes.

The Istio awareness module uses me.snowdrop:istio-client to interact with Istio APIs, letting us discover traffic rules, circuit breakers, and so on, making it easy for our Spring Boot applications to consume this data to dynamically configure themselves according to the environment.

Most of the components provided in this project need to know the namespace. For Kubernetes (1.3+), the namespace is made available to the pod as part of the service account secret and is automatically detected by the client. For earlier versions, it needs to be specified as an environment variable to the pod. A quick way to do this is as follows:

For distributions of Kubernetes that support more fine-grained role-based access within the cluster, you need to make sure a pod that runs with spring-cloud-kubernetes has access to the Kubernetes API. For any service accounts you assign to a deployment or pod, you need to make sure they have the correct roles.

Depending on the requirements, you’ll need get, list and watch permission on the following resources:

For development purposes, you can add cluster-reader permissions to your default service account. On a production system you’ll likely want to provide more granular permissions.

The following Role and RoleBinding are an example for namespaced permissions for the default account:

Below is a sample deployment YAML you can use to deploy the Kubernetes Configuration Watcher to Kubernetes.

The Service Account and associated Role Binding is important for Spring Cloud Kubernetes Configuration to work properly. The controller needs access to read data about ConfigMaps, Pods, Services, Endpoints and Secrets in the Kubernetes cluster.

If a change is made to a ConfigMap or Secret with valid labels (as detailed above), then Spring Cloud Kubernetes Configuration Watcher will take the name of the ConfigMap or Secret and send a notification to the application with that name. This might not be enough for your use-case though, you could for example want to:

For that reasons there is an addition annotation you could specify:

spring.cloud.kubernetes.configmap.apps or spring.cloud.kubernetes.secret.apps. It takes a String of apps separated by comma, that specifies the names of applications that will receive a notification when changes happen in this secret/configmap.

For example:

The HTTP implementation is what is used by default. When this implementation is used, Spring Cloud Kubernetes Configuration Watcher and a change to a ConfigMap or Secret occurs then the HTTP implementation will use the Spring Cloud Kubernetes Discovery Client to fetch all instances of the application which match the name of the ConfigMap or Secret and send an HTTP POST request to the application’s actuator /refresh endpoint. By default, it will send the post request to /actuator/refresh using the port registered in the discovery client.

The messaging implementation can be enabled by setting profile to either bus-amqp (RabbitMQ) or bus-kafka (Kafka) when the Spring Cloud Kubernetes Configuration Watcher application is deployed to Kubernetes.

When the bus-amqp profile is enabled you will need to configure Spring RabbitMQ to point it to the location of the RabbitMQ instance you would like to use as well as any credentials necessary to authenticate. This can be done by setting the standard Spring RabbitMQ properties, for example

When the bus-kafka profile is enabled you will need to configure Spring Kafka to point it to the location of the Kafka Broker instance you would like to use. This can be done by setting the standard Spring Kafka properties, for example

Below is a sample deployment, service and permissions configuration you can use to deploy a basic Config Server to Kubernetes.

The Spring Cloud Discovery server uses the Kubernetes API server to get data about Service and Endpoint resources so it needs list, watch, and get permissions to use those endpoints. See the below sample Kubernetes deployment YAML for an example of how to configure the Service Account on Kubernetes.

There are three endpoints exposed by the server.

An image of the Spring Cloud Discovery Server is hosted on Docker Hub. However, if you need to customize the discovery server behavior or prefer to build the image yourself you can easily build your own image from the source code on GitHub and use that.

Below is a sample deployment YAML you can use to deploy the Kubernetes Discovery Server to Kubernetes.

To build the source you will need to install JDK 17.

Spring Cloud uses Maven for most build-related activities, and you should be able to get off the ground quite quickly by cloning the project you are interested in and typing

The projects that require middleware (i.e. Redis) for testing generally require that a local instance of [Docker](www.docker.com/get-started) is installed and running.

The spring-cloud-build module has a "docs" profile, and if you switch that on it will try to build asciidoc sources from src/main/asciidoc. As part of that process it will look for a README.adoc and process it by loading all the includes, but not parsing or rendering it, just copying it to ${main.basedir} (defaults to $/opt/jenkins/data/workspace/spring-cloud-kubernetes-Kilburn-3.0.x-openjdk17-ci/docs, i.e. the root of the project). If there are any changes in the README it will then show up after a Maven build as a modified file in the correct place. Just commit it and push the change.

If you don’t have an IDE preference we would recommend that you use Spring Tools Suite or Eclipse when working with the code. We use the m2eclipse eclipse plugin for maven support. Other IDEs and tools should also work without issue as long as they use Maven 3.3.3 or better.

If you run the Spring Cloud Kuberentes build on an ARM64 machine the docker images used for the integration tests will fail to run due to using the wrong architecture. This is because the Paketo build pack does not yet support ARM64. To work around this you can run the build by passing -Dspring-boot.build-image.builder=dashaun/builder:tiny to Maven.

For example:

Before we accept a non-trivial patch or pull request we will need you to sign the Contributor License Agreement. Signing the contributor’s agreement does not grant anyone commit rights to the main repository, but it does mean that we can accept your contributions, and you will get an author credit if we do. Active contributors might be asked to join the core team, and given the ability to merge pull requests.

This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].

None of these is essential for a pull request, but they will all help. They can also be added after the original pull request but before a merge.

Spring Cloud Build comes with a set of checkstyle rules. You can find them in the spring-cloud-build-tools module. The most notable files under the module are:

Spring Cloud Build brings along the basepom:duplicate-finder-maven-plugin, that enables flagging duplicate and conflicting classes and resources on the java classpath.