This reference guide covers how to use Spring Cloud Kubernetes.
1. Why do you need Spring Cloud Kubernetes?
Spring Cloud Kubernetes provides implementations of well known Spring Cloud interfaces allowing developers to build and run Spring Cloud applications on Kubernetes. While this project may be useful to you when building a cloud native application, it is also not a requirement in order to deploy a Spring Boot app on Kubernetes. If you are just getting started in your journey to running your Spring Boot app on Kubernetes you can accomplish a lot with nothing more than a basic Spring Boot app and Kubernetes itself. To learn more, you can get started by reading the Spring Boot reference documentation for deploying to Kubernetes and also working through the workshop material Spring and Kubernetes.
2. Starters
Starters are convenient dependency descriptors you can include in your
application. Include a starter to get the dependencies and Spring Boot
auto-configuration for a feature set. Starters that begin with spring-cloud-starter-kubernetes-fabric8
provide implementations using the Fabric8 Kubernetes Java Client.
Starters that begin with
spring-cloud-starter-kubernetes-client
provide implementations using the Kubernetes Java Client.
Starter | Features |
---|---|
Fabric8 Dependency
Kubernetes Client Dependency
|
Discovery Client implementation that resolves service names to Kubernetes Services. |
Fabric8 Dependency
Kubernetes Client Dependency
|
Load application properties from Kubernetes ConfigMaps and Secrets. Reload application properties when a ConfigMap or Secret changes. |
Fabric8 Dependency
Kubernetes Client Dependency
|
All Spring Cloud Kubernetes features. |
3. DiscoveryClient for Kubernetes
This project provides an implementation of Discovery Client
for Kubernetes.
This client lets you query Kubernetes endpoints (see services) by name.
A service is typically exposed by the Kubernetes API server as a collection of endpoints that represent http
and https
addresses and that a client can
access from a Spring Boot application running as a pod.
DiscoveryClient can also find services of type ExternalName
(see ExternalName services). At the moment, external name support type of services is only available if the following property spring.cloud.kubernetes.discovery.include-external-name-services
is set to true
and only in the fabric8
implementation. In a later release, support will be added for the kubernetes native client also.
This is something that you get for free by adding the following dependency inside your project:
HTTP Based DiscoveryClient
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-discoveryclient</artifactId>
</dependency>
spring-cloud-starter-kubernetes-discoveryclient is designed to be used with the
Spring Cloud Kubernetes DiscoveryServer.
|
Fabric8 Kubernetes Client
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-fabric8</artifactId>
</dependency>
Kubernetes Java Client
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-client</artifactId>
</dependency>
To enable loading of the DiscoveryClient
, add @EnableDiscoveryClient
to the according configuration or application class, as the following example shows:
@SpringBootApplication
@EnableDiscoveryClient
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
Then you can inject the client in your code simply by autowiring it, as the following example shows:
@Autowired
private DiscoveryClient discoveryClient;
You can choose to enable DiscoveryClient
from all namespaces by setting the following property in application.properties
:
spring.cloud.kubernetes.discovery.all-namespaces=true
To discover services and endpoints only from specified namespaces you should set property all-namespaces
to false
and set the following property in application.properties
(in this example namespaces are: ns1
and ns2
).
spring.cloud.kubernetes.discovery.namespaces[0]=ns1
spring.cloud.kubernetes.discovery.namespaces[1]=ns2
To discover service endpoint addresses that are not marked as "ready" by the kubernetes api server, you can set the following property in application.properties
(default: false):
spring.cloud.kubernetes.discovery.include-not-ready-addresses=true
This might be useful when discovering services for monitoring purposes, and would enable inspecting the /health endpoint of not-ready service instances.
|
If your service exposes multiple ports, you will need to specify which port the DiscoveryClient
should use.
The DiscoveryClient
will choose the port using the following logic.
-
If the service has a label
primary-port-name
it will use the port with the name specified in the label’s value. -
If no label is present, then the port name specified in
spring.cloud.kubernetes.discovery.primary-port-name
will be used. -
If neither of the above are specified it will use the port named
https
. -
If none of the above conditions are met it will use the port named
http
. -
As a last resort it wil pick the first port in the list of ports.
The last option may result in non-deterministic behaviour. Please make sure to configure your service and/or application accordingly. |
By default all of the ports and their names will be added to the metadata of the ServiceInstance
.
As said before, if you want to get the list of ServiceInstance
to also include the ExternalName
type services, you need to enable that support via: spring.cloud.kubernetes.discovery.include-external-name-services=true
. As such, when calling DiscoveryClient::getInstances
those will be returned also. You can distinguish between ExternalName
and any other types by inspecting ServiceInstance::getMetadata
and lookup for a field called type
. This will be the type of the service returned : ExternalName
/ClusterIP
, etc.
ServiceInstance
can include the labels and annotations of specific pods from the underlying service instance. To obtain such information, you need to also enable:
spring.cloud.kubernetes.discovery.metadata.add-pod-labels=true
and/or spring.cloud.kubernetes.discovery.metadata.add-pod-annotations=true
. At the moment, such functionality is present only in the fabric8 client implementation, but will be added to the kubernetes native client in a later release.
If, for any reason, you need to disable the DiscoveryClient
, you can set the following property in application.properties
:
spring.cloud.kubernetes.discovery.enabled=false
Some Spring Cloud components use the DiscoveryClient
in order to obtain information about the local service instance. For
this to work, you need to align the Kubernetes service name with the spring.application.name
property.
spring.application.name has no effect as far as the name registered for the application within Kubernetes
|
Spring Cloud Kubernetes can also watch the Kubernetes service catalog for changes and update the
DiscoveryClient
implementation accordingly. By "watch" we mean that we will publish a heartbeat event every spring.cloud.kubernetes.discovery.catalog-services-watch-delay
milliseconds (by default it is 30000
). The heartbeat event will contain the target references (and their namespaces of the addresses of all endpoints
(for the exact details of what will get returned you can take a look inside KubernetesCatalogWatch
). This is an implementation detail, and listeners of the heartbeat event
should not rely on the details. Instead, they should see if there are differences between two subsequent heartbeats via equals
method. We will take care to return a correct implementation that adheres to the equals contract.
The endpoints will be queried in either :
-
all namespaces (enabled via
spring.cloud.kubernetes.discovery.all-namespaces=true
) -
specific namespaces (enabled via
spring.cloud.kubernetes.discovery.namespaces
), for example:
spring:
cloud:
kubernetes:
discovery:
namespaces:
- namespace-a
- namespace-b
-
we will use: Namespace Resolution if the above two paths are not taken.
In order to enable this functionality you need to add
@EnableScheduling
on a configuration class in your application.
By default, we use the Endpoints
(see kubernetes.io/docs/concepts/services-networking/service/#endpoints) API to find out the current state of services. There is another way though, via EndpointSlices
(kubernetes.io/docs/concepts/services-networking/endpoint-slices/). Such support can be enabled via a property: spring.cloud.kubernetes.discovery.use-endpoint-slices=true
(by default it is false
). Of course, your cluster has to support it also. As a matter of fact, if you enable this property, but your cluster does not support it, we will fail starting the application. If you decide to enable such support, you also need proper Role/ClusterRole set-up. For example:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: namespace-reader
rules:
- apiGroups: ["discovery.k8s.io"]
resources: ["endpointslices"]
verbs: ["get", "list", "watch"]
4. Kubernetes native service discovery
Kubernetes itself is capable of (server side) service discovery (see: kubernetes.io/docs/concepts/services-networking/service/#discovering-services). Using native kubernetes service discovery ensures compatibility with additional tooling, such as Istio (istio.io), a service mesh that is capable of load balancing, circuit breaker, failover, and much more.
The caller service then need only refer to names resolvable in a particular Kubernetes cluster. A simple implementation might use a spring RestTemplate
that refers to a fully qualified domain name (FQDN), such as {service-name}.{namespace}.svc.{cluster}.local:{service-port}
.
Additionally, you can use Hystrix for:
-
Circuit breaker implementation on the caller side, by annotating the spring boot application class with
@EnableCircuitBreaker
-
Fallback functionality, by annotating the respective method with
@HystrixCommand(fallbackMethod=
5. Kubernetes PropertySource implementations
The most common approach to configuring your Spring Boot application is to create an application.properties
or application.yaml
or
an application-profile.properties
or application-profile.yaml
file that contains key-value pairs that provide customization values to your
application or Spring Boot starters. You can override these properties by specifying system properties or environment
variables.
To enable this functionality you need to set spring.config.import=kubernetes:
in your application’s configuration properties.
Currently you can not specify a ConfigMap or Secret to load using spring.config.import
, by default Spring Cloud Kubernetes
will load a ConfigMap and/or Secret based on the spring.application.name
property. If spring.application.name
is not set it will
load a ConfigMap and/or Secret with the name application
.
If you would like to load Kubernetes PropertySource
s during the bootstrap phase like it worked prior to the 3.0.x release
you can either add spring-cloud-starter-bootstrap
to your application’s classpath or set spring.cloud.bootstrap.enabled=true
as an environment variable.
5.1. Using a ConfigMap
PropertySource
Kubernetes provides a resource named ConfigMap
to externalize the
parameters to pass to your application in the form of key-value pairs or embedded application.properties
or application.yaml
files.
The Spring Cloud Kubernetes Config project makes Kubernetes ConfigMap
instances available
during application startup and triggers hot reloading of beans or Spring context when changes are detected on
observed ConfigMap
instances.
Everything that follows is explained mainly referring to examples using ConfigMaps, but the same stands for Secrets, i.e.: every feature is supported for both.
The default behavior is to create a Fabric8ConfigMapPropertySource
(or a KubernetesClientConfigMapPropertySource
) based on a Kubernetes ConfigMap
that has a metadata.name
value of either the name of
your Spring application (as defined by its spring.application.name
property) or a custom name defined within the
application.properties
file under the following key: spring.cloud.kubernetes.config.name
.
However, more advanced configuration is possible where you can use multiple ConfigMap
instances.
The spring.cloud.kubernetes.config.sources
list makes this possible.
For example, you could define the following ConfigMap
instances:
spring:
application:
name: cloud-k8s-app
cloud:
kubernetes:
config:
name: default-name
namespace: default-namespace
sources:
# Spring Cloud Kubernetes looks up a ConfigMap named c1 in namespace default-namespace
- name: c1
# Spring Cloud Kubernetes looks up a ConfigMap named default-name in whatever namespace n2
- namespace: n2
# Spring Cloud Kubernetes looks up a ConfigMap named c3 in namespace n3
- namespace: n3
name: c3
In the preceding example, if spring.cloud.kubernetes.config.namespace
had not been set,
the ConfigMap
named c1
would be looked up in the namespace that the application runs.
See Namespace resolution to get a better understanding of how the namespace
of the application is resolved.
Any matching ConfigMap
that is found is processed as follows:
-
Apply individual configuration properties.
-
Apply as
yaml
(orproperties
) the content of any property that is named by the value ofspring.application.name
(if it’s not present, byapplication.yaml/properties
) -
Apply as a properties file the content of the above name + each active profile.
An example should make a lot more sense. Let’s suppose that spring.application.name=my-app
and that
we have a single active profile called k8s
. For a configuration as below:
kind: ConfigMap
apiVersion: v1
metadata:
name: my-app
data:
my-app.yaml: |-
...
my-app-k8s.yaml: |-
..
my-app-dev.yaml: |-
..
someProp: someValue
These is what we will end-up loading:
-
my-app.yaml
treated as a file -
my-app-k8s.yaml
treated as a file -
my-app-dev.yaml
ignored, sincedev
is not an active profile -
someProp: someValue
plain property
The single exception to the aforementioned flow is when the ConfigMap
contains a single key that indicates
the file is a YAML or properties file. In that case, the name of the key does NOT have to be application.yaml
or
application.properties
(it can be anything) and the value of the property is treated correctly.
This features facilitates the use case where the ConfigMap
was created by using something like the following:
kubectl create configmap game-config --from-file=/path/to/app-config.yaml
Assume that we have a Spring Boot application named demo
that uses the following properties to read its thread pool
configuration.
-
pool.size.core
-
pool.size.maximum
This can be externalized to config map in yaml
format as follows:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
pool.size.core: 1
pool.size.max: 16
Individual properties work fine for most cases. However, sometimes, embedded yaml
is more convenient. In this case, we
use a single property named application.yaml
to embed our yaml
, as follows:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
application.yaml: |-
pool:
size:
core: 1
max:16
The following example also works:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
custom-name.yaml: |-
pool:
size:
core: 1
max:16
You can also define the search to happen based on labels, for example:
spring:
application:
name: labeled-configmap-with-prefix
cloud:
kubernetes:
config:
enableApi: true
useNameAsPrefix: true
namespace: spring-k8s
sources:
- labels:
letter: a
This will search for every configmap in namespace spring-k8s
that has labels {letter : a}
. The important
thing to notice here is that unlike reading a configmap by name, this can result in multiple config maps read.
As usual, the same feature is supported for secrets.
You can also configure Spring Boot applications differently depending on active profiles that are merged together
when the ConfigMap
is read. You can provide different property values for different profiles by using an
application.properties
or application.yaml
property, specifying profile-specific values, each in their own document
(indicated by the ---
sequence), as follows:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
application.yml: |-
greeting:
message: Say Hello to the World
farewell:
message: Say Goodbye
---
spring:
profiles: development
greeting:
message: Say Hello to the Developers
farewell:
message: Say Goodbye to the Developers
---
spring:
profiles: production
greeting:
message: Say Hello to the Ops
In the preceding case, the configuration loaded into your Spring Application with the development
profile is as follows:
greeting:
message: Say Hello to the Developers
farewell:
message: Say Goodbye to the Developers
However, if the production
profile is active, the configuration becomes:
greeting:
message: Say Hello to the Ops
farewell:
message: Say Goodbye
If both profiles are active, the property that appears last within the ConfigMap
overwrites any preceding values.
Another option is to create a different config map per profile and spring boot will automatically fetch it based on active profiles
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
application.yml: |-
greeting:
message: Say Hello to the World
farewell:
message: Say Goodbye
kind: ConfigMap
apiVersion: v1
metadata:
name: demo-development
data:
application.yml: |-
spring:
profiles: development
greeting:
message: Say Hello to the Developers
farewell:
message: Say Goodbye to the Developers
kind: ConfigMap
apiVersion: v1
metadata:
name: demo-production
data:
application.yml: |-
spring:
profiles: production
greeting:
message: Say Hello to the Ops
farewell:
message: Say Goodbye
To tell Spring Boot which profile
should be enabled see the Spring Boot documentation.
One option for activating a specific profile when deploying to Kubernetes is to launch your Spring Boot application with an environment variable that you can define in the PodSpec at the container specification.
Deployment resource file, as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment-name
labels:
app: deployment-name
spec:
replicas: 1
selector:
matchLabels:
app: deployment-name
template:
metadata:
labels:
app: deployment-name
spec:
containers:
- name: container-name
image: your-image
env:
- name: SPRING_PROFILES_ACTIVE
value: "development"
You could run into a situation where there are multiple configs maps that have the same property names. For example:
kind: ConfigMap
apiVersion: v1
metadata:
name: config-map-one
data:
application.yml: |-
greeting:
message: Say Hello from one
and
kind: ConfigMap
apiVersion: v1
metadata:
name: config-map-two
data:
application.yml: |-
greeting:
message: Say Hello from two
Depending on the order in which you place these in bootstrap.yaml|properties
, you might end up with an un-expected result (the last config map wins). For example:
spring:
application:
name: cloud-k8s-app
cloud:
kubernetes:
config:
namespace: default-namespace
sources:
- name: config-map-two
- name: config-map-one
will result in property greetings.message
being Say Hello from one
.
There is a way to change this default configuration by specifying useNameAsPrefix
. For example:
spring:
application:
name: with-prefix
cloud:
kubernetes:
config:
useNameAsPrefix: true
namespace: default-namespace
sources:
- name: config-map-one
useNameAsPrefix: false
- name: config-map-two
Such a configuration will result in two properties being generated:
-
greetings.message
equal toSay Hello from one
. -
config-map-two.greetings.message
equal toSay Hello from two
Notice that spring.cloud.kubernetes.config.useNameAsPrefix
has a lower priority than spring.cloud.kubernetes.config.sources.useNameAsPrefix
.
This allows you to set a "default" strategy for all sources, at the same time allowing to override only a few.
If using the config map name is not an option, you can specify a different strategy, called : explicitPrefix
. Since this is an explicit prefix that
you select, it can only be supplied to the sources
level. At the same time it has a higher priority than useNameAsPrefix
. Let’s suppose we have a third config map with these entries:
kind: ConfigMap
apiVersion: v1
metadata:
name: config-map-three
data:
application.yml: |-
greeting:
message: Say Hello from three
A configuration like the one below:
spring:
application:
name: with-prefix
cloud:
kubernetes:
config:
useNameAsPrefix: true
namespace: default-namespace
sources:
- name: config-map-one
useNameAsPrefix: false
- name: config-map-two
explicitPrefix: two
- name: config-map-three
will result in three properties being generated:
-
greetings.message
equal toSay Hello from one
. -
two.greetings.message
equal toSay Hello from two
. -
config-map-three.greetings.message
equal toSay Hello from three
.
The same way you configure a prefix for configmaps, you can do it for secrets also; both for secrets that are based on name and the ones based on labels. For example:
spring:
application:
name: prefix-based-secrets
cloud:
kubernetes:
secrets:
enableApi: true
useNameAsPrefix: true
namespace: spring-k8s
sources:
- labels:
letter: a
useNameAsPrefix: false
- labels:
letter: b
explicitPrefix: two
- labels:
letter: c
- labels:
letter: d
useNameAsPrefix: true
- name: my-secret
The same processing rules apply when generating property source as for config maps. The only difference is that
potentially, looking up secrets by labels can mean that we find more than one source. In such a case, prefix (if specified via useNameAsPrefix
)
will be the names of all secrets found for those particular labels.
One more thing to bear in mind is that we support prefix
per source, not per secret. The easiest way to explain this is via an example:
spring:
application:
name: prefix-based-secrets
cloud:
kubernetes:
secrets:
enableApi: true
useNameAsPrefix: true
namespace: spring-k8s
sources:
- labels:
color: blue
useNameAsPrefix: true
Suppose that a query matching such a label will provide two secrets as a result: secret-a
and secret-b
.
Both of these secrets have the same property name: color=sea-blue
and color=ocean-blue
. It is undefined which
color
will end-up as part of property sources, but the prefix for it will be secret-a.secret-b
(concatenated sorted naturally, names of the secrets).
If you need more fine-grained results, adding more labels to identify the secret uniquely would be an option.
By default, besides reading the config map that is specified in the sources
configuration, Spring will also try to read
all properties from "profile aware" sources. The easiest way to explain this is via an example. Let’s suppose your application
enables a profile called "dev" and you have a configuration like the one below:
spring:
application:
name: spring-k8s
cloud:
kubernetes:
config:
namespace: default-namespace
sources:
- name: config-map-one
Besides reading the config-map-one
, Spring will also try to read config-map-one-dev
; in this particular order. Each active profile
generates such a profile aware config map.
Though your application should not be impacted by such a config map, it can be disabled if needed:
spring:
application:
name: spring-k8s
cloud:
kubernetes:
config:
includeProfileSpecificSources: false
namespace: default-namespace
sources:
- name: config-map-one
includeProfileSpecificSources: false
Notice that just like before, there are two levels where you can specify this property: for all config maps or for individual ones; the latter having a higher priority.
You should check the security configuration section. To access config maps from inside a pod you need to have the correct Kubernetes service accounts, roles and role bindings. |
Another option for using ConfigMap
instances is to mount them into the Pod by running the Spring Cloud Kubernetes application
and having Spring Cloud Kubernetes read them from the file system.
This feature is deprecated and will be removed in a future release (Use spring.config.import instead).
This behavior is controlled by the spring.cloud.kubernetes.config.paths property. You can use it in
addition to or instead of the mechanism described earlier.
spring.cloud.kubernetes.config.paths expects a List of full paths to each property file, because directories are not being recursively parsed. For example:
|
spring:
cloud:
kubernetes:
config:
paths:
- /tmp/application.properties
- /var/application.yaml
If you use spring.cloud.kubernetes.config.paths or spring.cloud.kubernetes.secrets.path the automatic reload
functionality will not work. You will need to make a POST request to the /actuator/refresh endpoint or
restart/redeploy the application.
|
In some cases, your application may be unable to load some of your ConfigMaps
using the Kubernetes API.
If you want your application to fail the start-up process in such cases, you can set
spring.cloud.kubernetes.config.fail-fast=true
to make the application start-up fail with an Exception.
You can also make your application retry loading ConfigMap
property sources on a failure. First, you need to
set spring.cloud.kubernetes.config.fail-fast=true
. Then you need to add spring-retry
and spring-boot-starter-aop
to your classpath. You can configure retry properties such as
the maximum number of attempts, backoff options like initial interval, multiplier, max interval by setting the
spring.cloud.kubernetes.config.retry.*
properties.
If you already have spring-retry and spring-boot-starter-aop on the classpath for some reason
and want to enable fail-fast, but do not want retry to be enabled; you can disable retry for ConfigMap PropertySources
by setting spring.cloud.kubernetes.config.retry.enabled=false .
|
Name | Type | Default | Description |
---|---|---|---|
|
|
|
Enable ConfigMaps |
|
|
|
Sets the name of |
|
|
Client namespace |
Sets the Kubernetes namespace where to lookup |
|
|
|
Sets the paths where |
|
|
|
Enable or disable consuming |
|
|
|
Enable or disable failing the application start-up when an error occurred while loading a |
|
|
|
Enable or disable config retry. |
|
|
|
Initial retry interval in milliseconds. |
|
|
|
Maximum number of attempts. |
|
|
|
Maximum interval for backoff. |
|
|
|
Multiplier for next interval. |
5.2. Secrets PropertySource
Kubernetes has the notion of Secrets for storing
sensitive data such as passwords, OAuth tokens, and so on. This project provides integration with Secrets
to make secrets
accessible by Spring Boot applications. You can explicitly enable or disable This feature by setting the spring.cloud.kubernetes.secrets.enabled
property.
When enabled, the Fabric8SecretsPropertySource
looks up Kubernetes for Secrets
from the following sources:
-
Reading recursively from secrets mounts
-
Named after the application (as defined by
spring.application.name
) -
Matching some labels
Note:
By default, consuming Secrets through the API (points 2 and 3 above) is not enabled for security reasons. The permission 'list' on secrets allows clients to inspect secrets values in the specified namespace. Further, we recommend that containers share secrets through mounted volumes.
If you enable consuming Secrets through the API, we recommend that you limit access to Secrets by using an authorization policy, such as RBAC. For more information about risks and best practices when consuming Secrets through the API refer to this doc.
If the secrets are found, their data is made available to the application.
Assume that we have a spring boot application named demo
that uses properties to read its database
configuration. We can create a Kubernetes secret by using the following command:
kubectl create secret generic db-secret --from-literal=username=user --from-literal=password=p455w0rd
The preceding command would create the following secret (which you can see by using kubectl get secrets db-secret -o yaml
):
apiVersion: v1
data:
password: cDQ1NXcwcmQ=
username: dXNlcg==
kind: Secret
metadata:
creationTimestamp: 2017-07-04T09:15:57Z
name: db-secret
namespace: default
resourceVersion: "357496"
selfLink: /api/v1/namespaces/default/secrets/db-secret
uid: 63c89263-6099-11e7-b3da-76d6186905a8
type: Opaque
Note that the data contains Base64-encoded versions of the literal provided by the create
command.
Your application can then use this secret — for example, by exporting the secret’s value as environment variables:
apiVersion: v1
kind: Deployment
metadata:
name: ${project.artifactId}
spec:
template:
spec:
containers:
- env:
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: db-secret
key: username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: password
You can select the Secrets to consume in a number of ways:
-
By listing the directories where secrets are mapped:
-Dspring.cloud.kubernetes.secrets.paths=/etc/secrets/db-secret,etc/secrets/postgresql
If you have all the secrets mapped to a common root, you can set them like:
-Dspring.cloud.kubernetes.secrets.paths=/etc/secrets
-
By setting a named secret:
-Dspring.cloud.kubernetes.secrets.name=db-secret
-
By defining a list of labels:
-Dspring.cloud.kubernetes.secrets.labels.broker=activemq -Dspring.cloud.kubernetes.secrets.labels.db=postgresql
As the case with ConfigMap
, more advanced configuration is also possible where you can use multiple Secret
instances. The spring.cloud.kubernetes.secrets.sources
list makes this possible.
For example, you could define the following Secret
instances:
spring:
application:
name: cloud-k8s-app
cloud:
kubernetes:
secrets:
name: default-name
namespace: default-namespace
sources:
# Spring Cloud Kubernetes looks up a Secret named s1 in namespace default-namespace
- name: s1
# Spring Cloud Kubernetes looks up a Secret named default-name in namespace n2
- namespace: n2
# Spring Cloud Kubernetes looks up a Secret named s3 in namespace n3
- namespace: n3
name: s3
In the preceding example, if spring.cloud.kubernetes.secrets.namespace
had not been set,
the Secret
named s1
would be looked up in the namespace that the application runs.
See namespace-resolution to get a better understanding of how the namespace
of the application is resolved.
Similar to the ConfigMaps
; if you want your application to fail to start
when it is unable to load Secrets
property sources, you can set spring.cloud.kubernetes.secrets.fail-fast=true
.
It is also possible to enable retry for Secret
property sources like the ConfigMaps
.
As with the ConfigMap
property sources, first you need to set spring.cloud.kubernetes.secrets.fail-fast=true
.
Then you need to add spring-retry
and spring-boot-starter-aop
to your classpath.
Retry behavior of the Secret
property sources can be configured by setting the spring.cloud.kubernetes.secrets.retry.*
properties.
If you already have spring-retry and spring-boot-starter-aop on the classpath for some reason
and want to enable fail-fast, but do not want retry to be enabled; you can disable retry for Secrets PropertySources
by setting spring.cloud.kubernetes.secrets.retry.enabled=false .
|
Name | Type | Default | Description |
---|---|---|---|
|
|
|
Enable Secrets |
|
|
|
Sets the name of the secret to look up |
|
|
Client namespace |
Sets the Kubernetes namespace where to look up |
|
|
|
Sets the labels used to lookup secrets |
|
|
|
Sets the paths where secrets are mounted (example 1) |
|
|
|
Enables or disables consuming secrets through APIs (examples 2 and 3) |
|
|
|
Enable or disable failing the application start-up when an error occurred while loading a |
|
|
|
Enable or disable secrets retry. |
|
|
|
Initial retry interval in milliseconds. |
|
|
|
Maximum number of attempts. |
|
|
|
Maximum interval for backoff. |
|
|
|
Multiplier for next interval. |
Notes:
-
The
spring.cloud.kubernetes.secrets.labels
property behaves as defined by Map-based binding. -
The
spring.cloud.kubernetes.secrets.paths
property behaves as defined by Collection-based binding. -
Access to secrets through the API may be restricted for security reasons. The preferred way is to mount secrets to the Pod.
You can find an example of an application that uses secrets (though it has not been updated to use the new spring-cloud-kubernetes
project) at
spring-boot-camel-config
5.3. Namespace resolution
Finding an application namespace happens on a best-effort basis. There are some steps that we iterate in order to find it. The easiest and most common one, is to specify it in the proper configuration, for example:
spring:
application:
name: app
cloud:
kubernetes:
secrets:
name: secret
namespace: default
sources:
# Spring Cloud Kubernetes looks up a Secret named 'a' in namespace 'default'
- name: a
# Spring Cloud Kubernetes looks up a Secret named 'secret' in namespace 'b'
- namespace: b
# Spring Cloud Kubernetes looks up a Secret named 'd' in namespace 'c'
- namespace: c
name: d
Remember that the same can be done for config maps. If such a namespace is not specified, it will be read (in this order):
-
from property
spring.cloud.kubernetes.client.namespace
-
from a String residing in a file denoted by
spring.cloud.kubernetes.client.serviceAccountNamespacePath
property -
from a String residing in
/var/run/secrets/kubernetes.io/serviceaccount/namespace
file (kubernetes default namespace path) -
from a designated client method call (for example fabric8’s :
KubernetesClient::getNamespace
), if the client provides such a method. This, in turn, could be configured via environment properties. For example fabric8 client can be configured via "KUBERNETES_NAMESPACE" property; consult the client documentation for exact details.
Failure to find a namespace from the above steps will result in an Exception being raised.
5.4. Order of ConfigMaps and Secrets
If, for whatever reason, you enabled both configmaps and secrets, and there is a common property between them, the value from the ConfigMap will have a higher precedence. That is: it will override whatever values are found in secrets.
5.5. PropertySource
Reload
This functionality has been deprecated in the 2020.0 release. Please see the Spring Cloud Kubernetes Configuration Watcher controller for an alternative way to achieve the same functionality. |
Some applications may need to detect changes on external property sources and update their internal status to reflect the new configuration.
The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related ConfigMap
or
Secret
changes.
By default, this feature is disabled. You can enable it by using the spring.cloud.kubernetes.reload.enabled=true
configuration property (for example, in the application.properties
file).
Please notice that this will enable monitoring of configmaps only (i.e.: spring.cloud.kubernetes.reload.monitoring-config-maps
will be set to true
).
If you want to enable monitoring of secrets, this must be done explicitly via : spring.cloud.kubernetes.reload.monitoring-secrets=true
.
The following levels of reload are supported (by setting the spring.cloud.kubernetes.reload.strategy
property):
-
refresh
(default): Only configuration beans annotated with@ConfigurationProperties
or@RefreshScope
are reloaded. This reload level leverages the refresh feature of Spring Cloud Context. -
restart_context
: the whole SpringApplicationContext
is gracefully restarted. Beans are recreated with the new configuration. In order for the restart context functionality to work properly you must enable and expose the restart actuator endpoint
management: endpoint: restart: enabled: true endpoints: web: exposure: include: restart
-
shutdown
: the SpringApplicationContext
is shut down to activate a restart of the container. When you use this level, make sure that the lifecycle of all non-daemon threads is bound to theApplicationContext
and that a replication controller or replica set is configured to restart the pod.
Assuming that the reload feature is enabled with default settings (refresh
mode), the following bean is refreshed when the config map changes:
@Configuration @ConfigurationProperties(prefix = "bean") public class MyConfig { private String message = "a message that can be changed live"; // getter and setters }
To see that changes effectively happen, you can create another bean that prints the message periodically, as follows
@Component
public class MyBean {
@Autowired
private MyConfig config;
@Scheduled(fixedDelay = 5000)
public void hello() {
System.out.println("The message is: " + config.getMessage());
}
}
You can change the message printed by the application by using a ConfigMap
, as follows:
apiVersion: v1
kind: ConfigMap
metadata:
name: reload-example
data:
application.properties: |-
bean.message=Hello World!
Any change to the property named bean.message
in the ConfigMap
associated with the pod is reflected in the
output. More generally speaking, changes associated to properties prefixed with the value defined by the prefix
field of the @ConfigurationProperties
annotation are detected and reflected in the application.
Associating a ConfigMap
with a pod is explained earlier in this chapter.
The full example is available in spring-cloud-kubernetes-reload-example
.
The reload feature supports two operating modes:
-
Event (default): Watches for changes in config maps or secrets by using the Kubernetes API (web socket). Any event produces a re-check on the configuration and, in case of changes, a reload. The
view
role on the service account is required in order to listen for config map changes. A higher level role (such asedit
) is required for secrets (by default, secrets are not monitored). -
Polling: Periodically re-creates the configuration from config maps and secrets to see if it has changed. You can configure the polling period by using the
spring.cloud.kubernetes.reload.period
property and defaults to 15 seconds. It requires the same role as the monitored property source. This means, for example, that using polling on file-mounted secret sources does not require particular privileges.
5.6. Reload namespace and label filtering
By default, a namespace chosen using the steps outlined in Namespace resolution will be used to listen to changes in configmaps and secrets. i.e.: if you do not tell reload what namespaces and configmaps/secrets to watch for, it will watch all configmaps/secrets from the namespace that will be computed using the above algorithm.
On the other hand, you can define a more fine-grained approach. For example, you can specify the namespaces where changes will be monitored:
spring:
application:
name: event-reload
cloud:
kubernetes:
reload:
enabled: true
strategy: shutdown
mode: event
namespaces:
- my-namespace
Such a configuration will make the app watch changes only in the my-namespace
namespace. Mind that this will
watch all configmaps/secrets (depending on which one you enable). If you want an even more fine-grained approach,
you can enable "label-filtering". First we need to enable such support via : enable-reload-filtering: true
spring:
application:
name: event-reload
cloud:
kubernetes:
reload:
enabled: true
strategy: shutdown
mode: event
namespaces:
- my-namespaces
monitoring-config-maps: true
enable-reload-filtering: true
What this will do, is watch configmaps/secrets that only have the spring.cloud.kubernetes.config.informer.enabled: true
label.
Name | Type | Default | Description |
---|---|---|---|
|
|
|
Enables monitoring of property sources and configuration reload |
|
|
|
Allow monitoring changes in config maps |
|
|
|
Allow monitoring changes in secrets |
|
|
|
The strategy to use when firing a reload ( |
|
|
|
Specifies how to listen for changes in property sources ( |
|
|
|
The period for verifying changes when using the |
|
|
namespaces where we should watch for changes |
|
|
|
enabled labeled filtering for reload functionality |
Notes:
-
You should not use properties under
spring.cloud.kubernetes.reload
in config maps or secrets. Changing such properties at runtime may lead to unexpected results. -
Deleting a property or the whole config map does not restore the original state of the beans when you use the
refresh
level.
6. Kubernetes Ecosystem Awareness
All features described earlier in this guide work equally well, regardless of whether your application is running inside
Kubernetes. This is really helpful for development and troubleshooting.
From a development point of view, this lets you start your Spring Boot application and debug one
of the modules that is part of this project. You need not deploy it in Kubernetes,
as the code of the project relies on the
Fabric8 Kubernetes Java client, which is a fluent DSL that can
communicate by using http
protocol to the REST API of the Kubernetes Server.
Kubernetes awareness is based on Spring Boot API, specifically on ConditionalOnCloudPlatform.
That property will auto-detect if your application is currently deployed in kubernetes or not. It is possible to override
that setting via spring.main.cloud-platform
.
For example, if you need to test some features, but do not want to deploy to a cluster, it is enough to set the:
spring.main.cloud-platform=KUBERNETES
. This will make spring-cloud-kubernetes
act as-if it is deployed in a real cluster.
If you have spring-cloud-starter-bootstrap on your classpath or are setting spring.cloud.bootstrap.enabled=true then
you will have to set spring.main.cloud-platform should be set in bootstrap.{properties|yml}
(or the profile specific one). Also note that these properties: spring.cloud.kubernetes.config.enabled and spring.cloud.kubernetes.secrets.enabled
will only take effect when set in bootstrap.{properties|yml} when you have spring-cloud-starter-bootstrap on your classpath or are setting spring.cloud.bootstrap.enabled=true .
|
6.1. Breaking Changes In 3.0.x
In versions of Spring Cloud Kubernetes prior to 3.0.x
, Kubernetes awareness was implemented using spring.cloud.kubernetes.enabled
property. This
property was removed and is un-supported. Instead, we use Spring Boot API: ConditionalOnCloudPlatform.
If it is needed to explicitly enable or disable this awareness, use spring.main.cloud-platform=NONE/KUBERNETES
.
6.2. Kubernetes Profile Autoconfiguration
When the application runs as a pod inside Kubernetes, a Spring profile named kubernetes
automatically gets activated.
This lets you customize the configuration, to define beans that are applied when the Spring Boot application is deployed
within the Kubernetes platform (for example, different development and production configuration).
6.3. Istio Awareness
When you include the spring-cloud-kubernetes-fabric8-istio
module in the application classpath, a new profile is added to the application,
provided the application is running inside a Kubernetes Cluster with Istio installed. You can then use
spring @Profile("istio")
annotations in your Beans and @Configuration
classes.
The Istio awareness module uses me.snowdrop:istio-client
to interact with Istio APIs, letting us discover traffic rules, circuit breakers, and so on,
making it easy for our Spring Boot applications to consume this data to dynamically configure themselves according to the environment.
7. Pod Health Indicator
Spring Boot uses HealthIndicator
to expose info about the health of an application.
That makes it really useful for exposing health-related information to the user and makes it a good fit for use as readiness probes.
The Kubernetes health indicator (which is part of the core module) exposes the following info:
-
Pod name, IP address, namespace, service account, node name, and its IP address
-
A flag that indicates whether the Spring Boot application is internal or external to Kubernetes
You can disable this HealthContributor
by setting management.health.kubernetes.enabled
to false
in application.[properties | yaml]
.
8. Info Contributor
Spring Cloud Kubernetes includes an InfoContributor
which adds Pod information to
Spring Boot’s /info
Acturator endpoint.
You can disable this InfoContributor
by setting management.info.kubernetes.enabled
to false
in application.[properties | yaml]
.
9. Leader Election
The Spring Cloud Kubernetes leader election mechanism implements the leader election API of Spring Integration using a Kubernetes ConfigMap.
Multiple application instances compete for leadership, but leadership will only be granted to one.
When granted leadership, a leader application receives an OnGrantedEvent
application event with leadership Context
.
Applications periodically attempt to gain leadership, with leadership granted to the first caller.
A leader will remain a leader until either it is removed from the cluster, or it yields its leadership.
When leadership removal occurs, the previous leader receives OnRevokedEvent
application event.
After removal, any instances in the cluster may become the new leader, including the old leader.
To include it in your project, add the following dependency.
Fabric8 Leader Implementation
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-kubernetes-fabric8-leader</artifactId>
</dependency>
To specify the name of the configmap used for leader election use the following property.
spring.cloud.kubernetes.leader.config-map-name=leader
10. LoadBalancer for Kubernetes
This project includes Spring Cloud Load Balancer for load balancing based on Kubernetes Endpoints and provides implementation of load balancer based on Kubernetes Service. To include it to your project add the following dependency.
Fabric8 Implementation
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-fabric8-loadbalancer</artifactId>
</dependency>
Kubernetes Java Client Implementation
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-client-loadbalancer</artifactId>
</dependency>
To enable load balancing based on Kubernetes Service name use the following property. Then load balancer would try to call application using address, for example service-a.default.svc.cluster.local
spring.cloud.kubernetes.loadbalancer.mode=SERVICE
To enabled load balancing across all namespaces use the following property. Property from spring-cloud-kubernetes-discovery
module is respected.
spring.cloud.kubernetes.discovery.all-namespaces=true
If a service needs to be accessed over HTTPS you need to add a label or annotation to your service definition with the name secured
and the value true
and the load balancer will then use HTTPS to make requests to the service.
11. Security Configurations Inside Kubernetes
11.1. Namespace
Most of the components provided in this project need to know the namespace. For Kubernetes (1.3+), the namespace is made available to the pod as part of the service account secret and is automatically detected by the client. For earlier versions, it needs to be specified as an environment variable to the pod. A quick way to do this is as follows:
env:
- name: "KUBERNETES_NAMESPACE"
valueFrom:
fieldRef:
fieldPath: "metadata.namespace"
11.2. Service Account
For distributions of Kubernetes that support more fine-grained role-based access within the cluster, you need to make sure a pod that runs with spring-cloud-kubernetes
has access to the Kubernetes API.
For any service accounts you assign to a deployment or pod, you need to make sure they have the correct roles.
Depending on the requirements, you’ll need get
, list
and watch
permission on the following resources:
Dependency | Resources |
---|---|
spring-cloud-starter-kubernetes-fabric8 |
pods, services, endpoints |
spring-cloud-starter-kubernetes-fabric8-config |
configmaps, secrets |
spring-cloud-starter-kubernetes-client |
pods, services, endpoints |
spring-cloud-starter-kubernetes-client-config |
configmaps, secrets |
For development purposes, you can add cluster-reader
permissions to your default
service account. On a production system you’ll likely want to provide more granular permissions.
The following Role and RoleBinding are an example for namespaced permissions for the default
account:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: YOUR-NAME-SPACE
name: namespace-reader
rules:
- apiGroups: [""]
resources: ["configmaps", "pods", "services", "endpoints", "secrets"]
verbs: ["get", "list", "watch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: namespace-reader-binding
namespace: YOUR-NAME-SPACE
subjects:
- kind: ServiceAccount
name: default
apiGroup: ""
roleRef:
kind: Role
name: namespace-reader
apiGroup: ""
12. Service Registry Implementation
In Kubernetes service registration is controlled by the platform, the application itself does not control
registration as it may do in other platforms. For this reason using spring.cloud.service-registry.auto-registration.enabled
or setting @EnableDiscoveryClient(autoRegister=false)
will have no effect in Spring Cloud Kubernetes.
13. Spring Cloud Kubernetes Configuration Watcher
Kubernetes provides the ability to mount a ConfigMap or Secret as a volume in the container of your application. When the contents of the ConfigMap or Secret changes, the mounted volume will be updated with those changes.
However, Spring Boot will not automatically update those changes unless you restart the application. Spring Cloud
provides the ability refresh the application context without restarting the application by either hitting the
actuator endpoint /refresh
or via publishing a RefreshRemoteApplicationEvent
using Spring Cloud Bus.
To achieve this configuration refresh of a Spring Cloud app running on Kubernetes, you can deploy the Spring Cloud Kubernetes Configuration Watcher controller into your Kubernetes cluster.
The application is published as a container and is available on Docker Hub. However, if you need to customize the config watcher behavior or prefer to build the image yourself you can easily build your own image from the source code on GitHub and use that.
Spring Cloud Kubernetes Configuration Watcher can send refresh notifications to applications in two ways.
-
Over HTTP in which case the application being notified must of the
/refresh
actuator endpoint exposed and accessible from within the cluster -
Using Spring Cloud Bus, in which case you will need a message broker deployed to your custer for the application to use.
13.1. Deployment YAML
Below is a sample deployment YAML you can use to deploy the Kubernetes Configuration Watcher to Kubernetes.
---
apiVersion: v1
kind: List
items:
- apiVersion: v1
kind: Service
metadata:
labels:
app: spring-cloud-kubernetes-configuration-watcher
name: spring-cloud-kubernetes-configuration-watcher
spec:
ports:
- name: http
port: 8888
targetPort: 8888
selector:
app: spring-cloud-kubernetes-configuration-watcher
type: ClusterIP
- apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: spring-cloud-kubernetes-configuration-watcher
name: spring-cloud-kubernetes-configuration-watcher
- apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app: spring-cloud-kubernetes-configuration-watcher
name: spring-cloud-kubernetes-configuration-watcher:view
roleRef:
kind: Role
apiGroup: rbac.authorization.k8s.io
name: namespace-reader
subjects:
- kind: ServiceAccount
name: spring-cloud-kubernetes-configuration-watcher
- apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: namespace-reader
rules:
- apiGroups: ["", "extensions", "apps"]
resources: ["configmaps", "pods", "services", "endpoints", "secrets"]
verbs: ["get", "list", "watch"]
- apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-cloud-kubernetes-configuration-watcher-deployment
spec:
selector:
matchLabels:
app: spring-cloud-kubernetes-configuration-watcher
template:
metadata:
labels:
app: spring-cloud-kubernetes-configuration-watcher
spec:
serviceAccount: spring-cloud-kubernetes-configuration-watcher
containers:
- name: spring-cloud-kubernetes-configuration-watcher
image: springcloud/spring-cloud-kubernetes-configuration-watcher:2.0.1-SNAPSHOT
imagePullPolicy: IfNotPresent
readinessProbe:
httpGet:
port: 8888
path: /actuator/health/readiness
livenessProbe:
httpGet:
port: 8888
path: /actuator/health/liveness
ports:
- containerPort: 8888
The Service Account and associated Role Binding is important for Spring Cloud Kubernetes Configuration to work properly. The controller needs access to read data about ConfigMaps, Pods, Services, Endpoints and Secrets in the Kubernetes cluster.
13.2. Monitoring ConfigMaps and Secrets
Spring Cloud Kubernetes Configuration Watcher will react to changes in ConfigMaps with a label of spring.cloud.kubernetes.config
with the value true
or any Secret with a label of spring.cloud.kubernetes.secret
with the value true
. If the ConfigMap or Secret does not have either of those labels
or the values of those labels is not true
then any changes will be ignored.
If a change is made to a ConfigMap or Secret with valid labels then Spring Cloud Kubernetes Configuration Watcher will take the name of the ConfigMap or Secret and send a notification to the application with that name. This might not be enough for your use-case though, you could for example what to:
-
bind a config-map to multiple applications, so that a change inside a single configmap triggers a refresh for many services
-
have profile based sources trigger events for your application
For that reasons there is an addition annotation you could specify:
spring.cloud.kubernetes.configmap.apps
or spring.cloud.kubernetes.secret.apps
. It takes a String of apps separated by comma,
that specifies the names of applications that will receive a notification when changes happen in this secret/configmap.
For example:
kind: ConfigMap
apiVersion: v1
metadata:
name: example-configmap
labels:
spring.cloud.kubernetes.config: "true"
annotations:
spring.cloud.kubernetes.configmap.apps: "app-a, app-b"
13.3. HTTP Implementation
The HTTP implementation is what is used by default. When this implementation is used Spring Cloud Kubernetes Configuration Watcher and a
change to a ConfigMap or Secret occurs then the HTTP implementation will use the Spring Cloud Kubernetes Discovery Client to fetch all
instances of the application which match the name of the ConfigMap or Secret and send an HTTP POST request to the application’s actuator
/refresh
endpoint. By default it will send the post request to /actuator/refresh
using the port registered in the discovery client.
13.3.1. Non-Default Management Port and Actuator Path
If the application is using a non-default actuator path and/or using a different port for the management endpoints, the Kubernetes service for the application
can add an annotation called boot.spring.io/actuator
and set its value to the path and port used by the application. For example
apiVersion: v1
kind: Service
metadata:
labels:
app: config-map-demo
name: config-map-demo
annotations:
boot.spring.io/actuator: http://:9090/myactuator/home
spec:
ports:
- name: http
port: 8080
targetPort: 8080
selector:
app: config-map-demo
Another way you can choose to configure the actuator path and/or management port is by setting
spring.cloud.kubernetes.configuration.watcher.actuatorPath
and spring.cloud.kubernetes.configuration.watcher.actuatorPort
.
13.4. Messaging Implementation
The messaging implementation can be enabled by setting profile to either bus-amqp
(RabbitMQ) or bus-kafka
(Kafka) when the Spring Cloud Kubernetes Configuration Watcher
application is deployed to Kubernetes.
13.5. Configuring RabbitMQ
When the bus-amqp
profile is enabled you will need to configure Spring RabbitMQ to point it to the location of the RabbitMQ
instance you would like to use as well as any credentials necessary to authenticate. This can be done
by setting the standard Spring RabbitMQ properties, for example
spring:
rabbitmq:
username: user
password: password
host: rabbitmq
13.6. Configuring Kafka
When the bus-kafka
profile is enabled you will need to configure Spring Kafka to point it to the location of the Kafka Broker
instance you would like to use. This can be done by setting the standard Spring Kafka properties, for example
spring:
kafka:
producer:
bootstrap-servers: localhost:9092
14. Spring Cloud Kubernetes Config Server
The Spring Cloud Kubernetes Config Server, is based on Spring Cloud Config Server and adds an environment repository for Kubernetes Config Maps and Secrets.
This is component is completely optional. However, it allows you to continue to leverage configuration you may have stored in existing environment repositories (Git, SVN, Vault, etc) with applications that you are running on Kubernetes.
A default image is located on Docker Hub which will allow you to easily get a Config Server deployed on Kubernetes without building the code and image yourself. However, if you need to customize the config server behavior or prefer to build the image yourself you can easily build your own image from the source code on GitHub and use that.
14.1. Configuration
14.1.1. Enabling The Kubernetes Environment Repository
To enable the Kubernetes environment repository the kubernetes
profile must be included in the list of active profiles.
You may activate other profiles as well to use other environment repository implementations.
14.1.2. Config Map and Secret PropertySources
By default, only Config Map data will be fetched. To enable Secrets as well you will need to set spring.cloud.kubernetes.secrets.enableApi=true
.
You can disable the Config Map PropertySource
by setting spring.cloud.kubernetes.config.enableApi=false
.
14.1.3. Fetching Config Map and Secret Data From Additional Namespaces
By default, the Kubernetes environment repository will only fetch Config Map and Secrets from the namespace in which it is deployed.
If you want to include data from other namespaces you can set spring.cloud.kubernetes.configserver.config-map-namespaces
and/or spring.cloud.kubernetes.configserver.secrets-namespaces
to a comma separated
list of namespace values.
If you set spring.cloud.kubernetes.configserver.config-map-namespaces and/or spring.cloud.kubernetes.configserver.secrets-namespaces
you will need to include the namespace in which the Config Server is deployed in order to continue to fetch Config Map and Secret data from that namespace.
|
14.1.4. Kubernetes Access Controls
The Kubernetes Config Server uses the Kubernetes API server to fetch Config Map and Secret data. In order for it to do that
it needs ability to get
and list
Config Map and Secrets (depending on what you enable/disable).
14.2. Deployment Yaml
Below is a sample deployment, service and permissions configuration you can use to deploy a basic Config Server to Kubernetes.
---
apiVersion: v1
kind: List
items:
- apiVersion: v1
kind: Service
metadata:
labels:
app: spring-cloud-kubernetes-configserver
name: spring-cloud-kubernetes-configserver
spec:
ports:
- name: http
port: 8888
targetPort: 8888
selector:
app: spring-cloud-kubernetes-configserver
type: ClusterIP
- apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: spring-cloud-kubernetes-configserver
name: spring-cloud-kubernetes-configserver
- apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app: spring-cloud-kubernetes-configserver
name: spring-cloud-kubernetes-configserver:view
roleRef:
kind: Role
apiGroup: rbac.authorization.k8s.io
name: namespace-reader
subjects:
- kind: ServiceAccount
name: spring-cloud-kubernetes-configserver
- apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: namespace-reader
rules:
- apiGroups: ["", "extensions", "apps"]
resources: ["configmaps", "secrets"]
verbs: ["get", "list"]
- apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-cloud-kubernetes-configserver-deployment
spec:
selector:
matchLabels:
app: spring-cloud-kubernetes-configserver
template:
metadata:
labels:
app: spring-cloud-kubernetes-configserver
spec:
serviceAccount: spring-cloud-kubernetes-configserver
containers:
- name: spring-cloud-kubernetes-configserver
image: springcloud/spring-cloud-kubernetes-configserver
imagePullPolicy: IfNotPresent
env:
- name: SPRING_PROFILES_INCLUDE
value: "kubernetes"
readinessProbe:
httpGet:
port: 8888
path: /actuator/health/readiness
livenessProbe:
httpGet:
port: 8888
path: /actuator/health/liveness
ports:
- containerPort: 8888
15. Spring Cloud Kubernetes Discovery Server
The Spring Cloud Kubernetes Discovery Server provides HTTP endpoints apps can use to gather information
about services available within a Kubernetes cluster. The Spring Cloud Kubernetes Discovery Server
can be used by apps using the spring-cloud-starter-kubernetes-discoveryclient
to provide data to
the DiscoveryClient
implementation provided by that starter.
15.1. Permissions
The Spring Cloud Discovery server uses the Kubernetes API server to get data about Service and Endpoint resrouces so it needs list, watch, and get permissions to use those endpoints. See the below sample Kubernetes deployment YAML for an examlpe of how to configure the Service Account on Kubernetes.
15.2. Endpoints
There are three endpoints exposed by the server.
15.2.1. /apps
A GET
request sent to /apps
will return a JSON array of available services. Each item contains
the name of the Kubernetes service and service instance information. Below is a sample response.
[
{
"name":"spring-cloud-kubernetes-discoveryserver",
"serviceInstances":[
{
"instanceId":"836a2f25-daee-4af2-a1be-aab9ce2b938f",
"serviceId":"spring-cloud-kubernetes-discoveryserver",
"host":"10.244.1.6",
"port":8761,
"uri":"http://10.244.1.6:8761",
"secure":false,
"metadata":{
"app":"spring-cloud-kubernetes-discoveryserver",
"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"spring-cloud-kubernetes-discoveryserver\"},\"name\":\"spring-cloud-kubernetes-discoveryserver\",\"namespace\":\"default\"},\"spec\":{\"ports\":[{\"name\":\"http\",\"port\":80,\"targetPort\":8761}],\"selector\":{\"app\":\"spring-cloud-kubernetes-discoveryserver\"},\"type\":\"ClusterIP\"}}\n",
"http":"8761"
},
"namespace":"default",
"scheme":"http"
}
]
},
{
"name":"kubernetes",
"serviceInstances":[
{
"instanceId":"1234",
"serviceId":"kubernetes",
"host":"172.18.0.3",
"port":6443,
"uri":"http://172.18.0.3:6443",
"secure":false,
"metadata":{
"provider":"kubernetes",
"component":"apiserver",
"https":"6443"
},
"namespace":"default",
"scheme":"http"
}
]
}
]
15.2.2. /apps/{name}
A GET
request to /apps/{name}
can be used to get instance data for all instances of a given
service. Below is a sample response when a GET
request is made to /apps/kubernetes
.
[
{
"instanceId":"1234",
"serviceId":"kubernetes",
"host":"172.18.0.3",
"port":6443,
"uri":"http://172.18.0.3:6443",
"secure":false,
"metadata":{
"provider":"kubernetes",
"component":"apiserver",
"https":"6443"
},
"namespace":"default",
"scheme":"http"
}
]
15.2.3. /app/{name}/{instanceid}
A GET
request made to /app/{name}/{instanceid}
will return the instance data for a specific
instance of a given service. Below is a sample response when a GET
request is made to /app/kubernetes/1234
.
{
"instanceId":"1234",
"serviceId":"kubernetes",
"host":"172.18.0.3",
"port":6443,
"uri":"http://172.18.0.3:6443",
"secure":false,
"metadata":{
"provider":"kubernetes",
"component":"apiserver",
"https":"6443"
},
"namespace":"default",
"scheme":"http"
}
15.3. Deployment YAML
An image of the Spring Cloud Discovery Server is hosted on Docker Hub. However, if you need to customize the discovery server behavior or prefer to build the image yourself you can easily build your own image from the source code on GitHub and use that.
Below is a sample deployment YAML you can use to deploy the Kubernetes Configuration Watcher to Kubernetes.
---
apiVersion: v1
kind: List
items:
- apiVersion: v1
kind: Service
metadata:
labels:
app: spring-cloud-kubernetes-discoveryserver
name: spring-cloud-kubernetes-discoveryserver
spec:
ports:
- name: http
port: 80
targetPort: 8761
selector:
app: spring-cloud-kubernetes-discoveryserver
type: ClusterIP
- apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app: spring-cloud-kubernetes-discoveryserver
name: spring-cloud-kubernetes-discoveryserver
- apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app: spring-cloud-kubernetes-discoveryserver
name: spring-cloud-kubernetes-discoveryserver:view
roleRef:
kind: Role
apiGroup: rbac.authorization.k8s.io
name: namespace-reader
subjects:
- kind: ServiceAccount
name: spring-cloud-kubernetes-discoveryserver
- apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: namespace-reader
rules:
- apiGroups: ["", "extensions", "apps"]
resources: ["services", "endpoints"]
verbs: ["get", "list", "watch"]
- apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-cloud-kubernetes-discoveryserver-deployment
spec:
selector:
matchLabels:
app: spring-cloud-kubernetes-discoveryserver
template:
metadata:
labels:
app: spring-cloud-kubernetes-discoveryserver
spec:
serviceAccount: spring-cloud-kubernetes-discoveryserver
containers:
- name: spring-cloud-kubernetes-discoveryserver
image: springcloud/spring-cloud-kubernetes-discoveryserver:3.0.0-SNAPSHOT
imagePullPolicy: IfNotPresent
readinessProbe:
httpGet:
port: 8761
path: /actuator/health/readiness
livenessProbe:
httpGet:
port: 8761
path: /actuator/health/liveness
ports:
- containerPort: 8761
16. Examples
Spring Cloud Kubernetes tries to make it transparent for your applications to consume Kubernetes Native Services by following the Spring Cloud interfaces.
In your applications, you need to add the spring-cloud-kubernetes-discovery
dependency to your classpath and remove any other dependency that contains a DiscoveryClient
implementation (that is, a Eureka discovery client).
The same applies for PropertySourceLocator
, where you need to add to the classpath the spring-cloud-kubernetes-config
and remove any other dependency that contains a PropertySourceLocator
implementation (that is, a configuration server client).
The following projects highlight the usage of these dependencies and demonstrate how you can use these libraries from any Spring Boot application:
-
Spring Cloud Kubernetes Examples: the ones located inside this repository.
-
Spring Cloud Kubernetes Full Example: Minions and Boss
-
Spring Cloud Kubernetes Full Example: SpringOne Platform Tickets Service
-
Spring Cloud Gateway with Spring Cloud Kubernetes Discovery and Config
-
Spring Boot Admin with Spring Cloud Kubernetes Discovery and Config
17. Other Resources
This section lists other resources, such as presentations (slides) and videos about Spring Cloud Kubernetes.
Please feel free to submit other resources through pull requests to this repository.
18. Configuration properties
To see the list of all Kubernetes related configuration properties please check the Appendix page.
19. Building
19.1. Basic Compile and Test
To build the source you will need to install JDK 17.
Spring Cloud uses Maven for most build-related activities, and you should be able to get off the ground quite quickly by cloning the project you are interested in and typing
$ ./mvnw install
You can also install Maven (>=3.3.3) yourself and run the mvn command
in place of ./mvnw in the examples below. If you do that you also
might need to add -P spring if your local Maven settings do not
contain repository declarations for spring pre-release artifacts.
|
Be aware that you might need to increase the amount of memory
available to Maven by setting a MAVEN_OPTS environment variable with
a value like -Xmx512m -XX:MaxPermSize=128m . We try to cover this in
the .mvn configuration, so if you find you have to do it to make a
build succeed, please raise a ticket to get the settings added to
source control.
|
The projects that require middleware (i.e. Redis) for testing generally require that a local instance of [Docker](www.docker.com/get-started) is installed and running.
19.2. Documentation
The spring-cloud-build module has a "docs" profile, and if you switch
that on it will try to build asciidoc sources from
src/main/asciidoc
. As part of that process it will look for a
README.adoc
and process it by loading all the includes, but not
parsing or rendering it, just copying it to ${main.basedir}
(defaults to $/tmp/releaser-1690493683873-0/spring-cloud-kubernetes/docs
, i.e. the root of the project). If there are
any changes in the README it will then show up after a Maven build as
a modified file in the correct place. Just commit it and push the change.
19.3. Working with the code
If you don’t have an IDE preference we would recommend that you use Spring Tools Suite or Eclipse when working with the code. We use the m2eclipse eclipse plugin for maven support. Other IDEs and tools should also work without issue as long as they use Maven 3.3.3 or better.
19.3.1. Activate the Spring Maven profile
Spring Cloud projects require the 'spring' Maven profile to be activated to resolve the spring milestone and snapshot repositories. Use your preferred IDE to set this profile to be active, or you may experience build errors.
19.3.2. Importing into eclipse with m2eclipse
We recommend the m2eclipse eclipse plugin when working with eclipse. If you don’t already have m2eclipse installed it is available from the "eclipse marketplace".
Older versions of m2e do not support Maven 3.3, so once the
projects are imported into Eclipse you will also need to tell
m2eclipse to use the right profile for the projects. If you
see many different errors related to the POMs in the projects, check
that you have an up to date installation. If you can’t upgrade m2e,
add the "spring" profile to your settings.xml . Alternatively you can
copy the repository settings from the "spring" profile of the parent
pom into your settings.xml .
|
19.3.3. Importing into eclipse without m2eclipse
If you prefer not to use m2eclipse you can generate eclipse project metadata using the following command:
$ ./mvnw eclipse:eclipse
The generated eclipse projects can be imported by selecting import existing projects
from the file
menu.
19.4. Building Docker Images On ARM64
If you run the Spring Cloud Kuberentes build on an ARM64 machine the docker images
used for the integration tests will fail to run due to using the wrong architecture.
This is because the Paketo build pack does not yet support ARM64. To work around this you
can run the build by passing -Dspring-boot.build-image.builder=dashaun/builder:tiny
to Maven.
For example:
./mvnw clean install -Dspring-boot.build-image.builder=dashaun/builder:tiny
20. Contributing
Spring Cloud is released under the non-restrictive Apache 2.0 license, and follows a very standard Github development process, using Github tracker for issues and merging pull requests into master. If you want to contribute even something trivial please do not hesitate, but follow the guidelines below.
20.1. Sign the Contributor License Agreement
Before we accept a non-trivial patch or pull request we will need you to sign the Contributor License Agreement. Signing the contributor’s agreement does not grant anyone commit rights to the main repository, but it does mean that we can accept your contributions, and you will get an author credit if we do. Active contributors might be asked to join the core team, and given the ability to merge pull requests.
20.2. Code of Conduct
This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
20.3. Code Conventions and Housekeeping
None of these is essential for a pull request, but they will all help. They can also be added after the original pull request but before a merge.
-
Use the Spring Framework code format conventions. If you use Eclipse you can import formatter settings using the
eclipse-code-formatter.xml
file from the Spring Cloud Build project. If using IntelliJ, you can use the Eclipse Code Formatter Plugin to import the same file. -
Make sure all new
.java
files to have a simple Javadoc class comment with at least an@author
tag identifying you, and preferably at least a paragraph on what the class is for. -
Add the ASF license header comment to all new
.java
files (copy from existing files in the project) -
Add yourself as an
@author
to the .java files that you modify substantially (more than cosmetic changes). -
Add some Javadocs and, if you change the namespace, some XSD doc elements.
-
A few unit tests would help a lot as well — someone has to do it.
-
If no-one else is using your branch, please rebase it against the current master (or other target branch in the main project).
-
When writing a commit message please follow these conventions, if you are fixing an existing issue please add
Fixes gh-XXXX
at the end of the commit message (where XXXX is the issue number).
20.4. Checkstyle
Spring Cloud Build comes with a set of checkstyle rules. You can find them in the spring-cloud-build-tools
module. The most notable files under the module are:
└── src ├── checkstyle │ └── checkstyle-suppressions.xml (3) └── main └── resources ├── checkstyle-header.txt (2) └── checkstyle.xml (1)
1 | Default Checkstyle rules |
2 | File header setup |
3 | Default suppression rules |
20.4.1. Checkstyle configuration
Checkstyle rules are disabled by default. To add checkstyle to your project just define the following properties and plugins.
<properties> <maven-checkstyle-plugin.failsOnError>true</maven-checkstyle-plugin.failsOnError> (1) <maven-checkstyle-plugin.failsOnViolation>true </maven-checkstyle-plugin.failsOnViolation> (2) <maven-checkstyle-plugin.includeTestSourceDirectory>true </maven-checkstyle-plugin.includeTestSourceDirectory> (3) </properties> <build> <plugins> <plugin> (4) <groupId>io.spring.javaformat</groupId> <artifactId>spring-javaformat-maven-plugin</artifactId> </plugin> <plugin> (5) <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-checkstyle-plugin</artifactId> </plugin> </plugins> <reporting> <plugins> <plugin> (5) <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-checkstyle-plugin</artifactId> </plugin> </plugins> </reporting> </build>
1 | Fails the build upon Checkstyle errors |
2 | Fails the build upon Checkstyle violations |
3 | Checkstyle analyzes also the test sources |
4 | Add the Spring Java Format plugin that will reformat your code to pass most of the Checkstyle formatting rules |
5 | Add checkstyle plugin to your build and reporting phases |
If you need to suppress some rules (e.g. line length needs to be longer), then it’s enough for you to define a file under ${project.root}/src/checkstyle/checkstyle-suppressions.xml
with your suppressions. Example:
<?xml version="1.0"?> <!DOCTYPE suppressions PUBLIC "-//Puppy Crawl//DTD Suppressions 1.1//EN" "https://www.puppycrawl.com/dtds/suppressions_1_1.dtd"> <suppressions> <suppress files=".*ConfigServerApplication\.java" checks="HideUtilityClassConstructor"/> <suppress files=".*ConfigClientWatch\.java" checks="LineLengthCheck"/> </suppressions>
It’s advisable to copy the ${spring-cloud-build.rootFolder}/.editorconfig
and ${spring-cloud-build.rootFolder}/.springformat
to your project. That way, some default formatting rules will be applied. You can do so by running this script:
$ curl https://raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/.editorconfig -o .editorconfig
$ touch .springformat
20.5. IDE setup
20.5.1. Intellij IDEA
In order to setup Intellij you should import our coding conventions, inspection profiles and set up the checkstyle plugin. The following files can be found in the Spring Cloud Build project.
└── src ├── checkstyle │ └── checkstyle-suppressions.xml (3) └── main └── resources ├── checkstyle-header.txt (2) ├── checkstyle.xml (1) └── intellij ├── Intellij_Project_Defaults.xml (4) └── Intellij_Spring_Boot_Java_Conventions.xml (5)
1 | Default Checkstyle rules |
2 | File header setup |
3 | Default suppression rules |
4 | Project defaults for Intellij that apply most of Checkstyle rules |
5 | Project style conventions for Intellij that apply most of Checkstyle rules |
Go to File
→ Settings
→ Editor
→ Code style
. There click on the icon next to the Scheme
section. There, click on the Import Scheme
value and pick the Intellij IDEA code style XML
option. Import the spring-cloud-build-tools/src/main/resources/intellij/Intellij_Spring_Boot_Java_Conventions.xml
file.
Go to File
→ Settings
→ Editor
→ Inspections
. There click on the icon next to the Profile
section. There, click on the Import Profile
and import the spring-cloud-build-tools/src/main/resources/intellij/Intellij_Project_Defaults.xml
file.
To have Intellij work with Checkstyle, you have to install the Checkstyle
plugin. It’s advisable to also install the Assertions2Assertj
to automatically convert the JUnit assertions
Go to File
→ Settings
→ Other settings
→ Checkstyle
. There click on the +
icon in the Configuration file
section. There, you’ll have to define where the checkstyle rules should be picked from. In the image above, we’ve picked the rules from the cloned Spring Cloud Build repository. However, you can point to the Spring Cloud Build’s GitHub repository (e.g. for the checkstyle.xml
: raw.githubusercontent.com/spring-cloud/spring-cloud-build/master/spring-cloud-build-tools/src/main/resources/checkstyle.xml
). We need to provide the following variables:
-
checkstyle.header.file
- please point it to the Spring Cloud Build’s,spring-cloud-build-tools/src/main/resources/checkstyle-header.txt
file either in your cloned repo or via theraw.githubusercontent.com/spring-cloud/spring-cloud-build/master/spring-cloud-build-tools/src/main/resources/checkstyle-header.txt
URL. -
checkstyle.suppressions.file
- default suppressions. Please point it to the Spring Cloud Build’s,spring-cloud-build-tools/src/checkstyle/checkstyle-suppressions.xml
file either in your cloned repo or via theraw.githubusercontent.com/spring-cloud/spring-cloud-build/master/spring-cloud-build-tools/src/checkstyle/checkstyle-suppressions.xml
URL. -
checkstyle.additional.suppressions.file
- this variable corresponds to suppressions in your local project. E.g. you’re working onspring-cloud-contract
. Then point to theproject-root/src/checkstyle/checkstyle-suppressions.xml
folder. Example forspring-cloud-contract
would be:/home/username/spring-cloud-contract/src/checkstyle/checkstyle-suppressions.xml
.
Remember to set the Scan Scope to All sources since we apply checkstyle rules for production and test sources.
|
20.6. Duplicate Finder
Spring Cloud Build brings along the basepom:duplicate-finder-maven-plugin
, that enables flagging duplicate and conflicting classes and resources on the java classpath.
20.6.1. Duplicate Finder configuration
Duplicate finder is enabled by default and will run in the verify
phase of your Maven build, but it will only take effect in your project if you add the duplicate-finder-maven-plugin
to the build
section of the projecst’s pom.xml
.
<build>
<plugins>
<plugin>
<groupId>org.basepom.maven</groupId>
<artifactId>duplicate-finder-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
For other properties, we have set defaults as listed in the plugin documentation.
You can easily override them but setting the value of the selected property prefixed with duplicate-finder-maven-plugin
. For example, set duplicate-finder-maven-plugin.skip
to true
in order to skip duplicates check in your build.
If you need to add ignoredClassPatterns
or ignoredResourcePatterns
to your setup, make sure to add them in the plugin configuration section of your project:
<build>
<plugins>
<plugin>
<groupId>org.basepom.maven</groupId>
<artifactId>duplicate-finder-maven-plugin</artifactId>
<configuration>
<ignoredClassPatterns>
<ignoredClassPattern>org.joda.time.base.BaseDateTime</ignoredClassPattern>
<ignoredClassPattern>.*module-info</ignoredClassPattern>
</ignoredClassPatterns>
<ignoredResourcePatterns>
<ignoredResourcePattern>changelog.txt</ignoredResourcePattern>
</ignoredResourcePatterns>
</configuration>
</plugin>
</plugins>
</build>
21. AOT and native image support
At this point, Spring Cloud Kubernetes does not support Spring Boot AOT transformations or native images. Partial support might be added in future releases.