Spring Boot includes a number of additional features to help you monitor and manage your application when you push it to production. You can choose to manage and monitor your application by using HTTP endpoints or with JMX. Auditing, health, and metrics gathering can also be automatically applied to your application.
1. Enabling Production-ready Features
The spring-boot-actuator
module provides all of Spring Boot’s production-ready features.
The recommended way to enable the features is to add a dependency on the spring-boot-starter-actuator
‘Starter’.
To add the actuator to a Maven based project, add the following ‘Starter’ dependency:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>
For Gradle, use the following declaration:
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-actuator'
}
2. Endpoints
Actuator endpoints let you monitor and interact with your application.
Spring Boot includes a number of built-in endpoints and lets you add your own.
For example, the health
endpoint provides basic application health information.
Each individual endpoint can be enabled or disabled and exposed (made remotely accessible) over HTTP or JMX.
An endpoint is considered to be available when it is both enabled and exposed.
The built-in endpoints will only be auto-configured when they are available.
Most applications choose exposure via HTTP, where the ID of the endpoint along with a prefix of /actuator
is mapped to a URL.
For example, by default, the health
endpoint is mapped to /actuator/health
.
To learn more about the Actuator’s endpoints and their request and response formats, please refer to the separate API documentation (HTML or PDF). |
The following technology-agnostic endpoints are available:
ID | Description |
---|---|
|
Exposes audit events information for the current application.
Requires an |
|
Displays a complete list of all the Spring beans in your application. |
|
Exposes available caches. |
|
Shows the conditions that were evaluated on configuration and auto-configuration classes and the reasons why they did or did not match. |
|
Displays a collated list of all |
|
Exposes properties from Spring’s |
|
Shows any Flyway database migrations that have been applied.
Requires one or more |
|
Shows application health information. |
|
Displays HTTP trace information (by default, the last 100 HTTP request-response exchanges).
Requires an |
|
Displays arbitrary application info. |
|
Shows the Spring Integration graph.
Requires a dependency on |
|
Shows and modifies the configuration of loggers in the application. |
|
Shows any Liquibase database migrations that have been applied.
Requires one or more |
|
Shows ‘metrics’ information for the current application. |
|
Displays a collated list of all |
|
Shows information about Quartz Scheduler jobs. |
|
Displays the scheduled tasks in your application. |
|
Allows retrieval and deletion of user sessions from a Spring Session-backed session store. Requires a Servlet-based web application using Spring Session. |
|
Lets the application be gracefully shutdown. Disabled by default. |
|
Shows the startup steps data collected by the |
|
Performs a thread dump. |
If your application is a web application (Spring MVC, Spring WebFlux, or Jersey), you can use the following additional endpoints:
ID | Description |
---|---|
|
Returns an |
|
Exposes JMX beans over HTTP (when Jolokia is on the classpath, not available for WebFlux).
Requires a dependency on |
|
Returns the contents of the logfile (if |
|
Exposes metrics in a format that can be scraped by a Prometheus server.
Requires a dependency on |
2.1. Enabling Endpoints
By default, all endpoints except for shutdown
are enabled.
To configure the enablement of an endpoint, use its management.endpoint.<id>.enabled
property.
The following example enables the shutdown
endpoint:
management.endpoint.shutdown.enabled=true
management:
endpoint:
shutdown:
enabled: true
If you prefer endpoint enablement to be opt-in rather than opt-out, set the management.endpoints.enabled-by-default
property to false
and use individual endpoint enabled
properties to opt back in.
The following example enables the info
endpoint and disables all other endpoints:
management.endpoints.enabled-by-default=false
management.endpoint.info.enabled=true
management:
endpoints:
enabled-by-default: false
endpoint:
info:
enabled: true
Disabled endpoints are removed entirely from the application context.
If you want to change only the technologies over which an endpoint is exposed, use the include and exclude properties instead.
|
2.2. Exposing Endpoints
Since Endpoints may contain sensitive information, careful consideration should be given about when to expose them. The following table shows the default exposure for the built-in endpoints:
ID | JMX | Web |
---|---|---|
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
Yes |
|
N/A |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
N/A |
No |
|
N/A |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
N/A |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
|
Yes |
No |
To change which endpoints are exposed, use the following technology-specific include
and exclude
properties:
Property | Default |
---|---|
|
|
|
|
|
|
|
|
The include
property lists the IDs of the endpoints that are exposed.
The exclude
property lists the IDs of the endpoints that should not be exposed.
The exclude
property takes precedence over the include
property.
Both include
and exclude
properties can be configured with a list of endpoint IDs.
For example, to stop exposing all endpoints over JMX and only expose the health
and info
endpoints, use the following property:
management.endpoints.jmx.exposure.include=health,info
management:
endpoints:
jmx:
exposure:
include: "health,info"
*
can be used to select all endpoints.
For example, to expose everything over HTTP except the env
and beans
endpoints, use the following properties:
management.endpoints.web.exposure.include=*
management.endpoints.web.exposure.exclude=env,beans
management:
endpoints:
web:
exposure:
include: "*"
exclude: "env,beans"
* has a special meaning in YAML, so be sure to add quotes if you want to include (or exclude) all endpoints.
|
If your application is exposed publicly, we strongly recommend that you also secure your endpoints. |
If you want to implement your own strategy for when endpoints are exposed, you can register an EndpointFilter bean.
|
2.3. Securing HTTP Endpoints
You should take care to secure HTTP endpoints in the same way that you would any other sensitive URL.
If Spring Security is present, endpoints are secured by default using Spring Security’s content-negotiation strategy.
If you wish to configure custom security for HTTP endpoints, for example, only allow users with a certain role to access them, Spring Boot provides some convenient RequestMatcher
objects that can be used in combination with Spring Security.
A typical Spring Security configuration might look something like the following example:
@Configuration(proxyBeanMethods = false)
public class MySecurityConfiguration {
@Bean
public SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception {
http.requestMatcher(EndpointRequest.toAnyEndpoint())
.authorizeRequests((requests) -> requests.anyRequest().hasRole("ENDPOINT_ADMIN"));
http.httpBasic();
return http.build();
}
}
The preceding example uses EndpointRequest.toAnyEndpoint()
to match a request to any endpoint and then ensures that all have the ENDPOINT_ADMIN
role.
Several other matcher methods are also available on EndpointRequest
.
See the API documentation (HTML or PDF) for details.
If you deploy applications behind a firewall, you may prefer that all your actuator endpoints can be accessed without requiring authentication.
You can do so by changing the management.endpoints.web.exposure.include
property, as follows:
management.endpoints.web.exposure.include=*
management:
endpoints:
web:
exposure:
include: "*"
Additionally, if Spring Security is present, you would need to add custom security configuration that allows unauthenticated access to the endpoints as shown in the following example:
@Configuration(proxyBeanMethods = false)
public class MySecurityConfiguration {
@Bean
public SecurityFilterChain securityFilterChain(HttpSecurity http) throws Exception {
http.requestMatcher(EndpointRequest.toAnyEndpoint())
.authorizeRequests((requests) -> requests.anyRequest().permitAll());
return http.build();
}
}
In both the examples above, the configuration applies only to the actuator endpoints.
Since Spring Boot’s security configuration backs off completely in the presence of any SecurityFilterChain bean, you will need to configure an additional SecurityFilterChain bean with rules that apply to the rest of the application.
|
2.4. Configuring Endpoints
Endpoints automatically cache responses to read operations that do not take any parameters.
To configure the amount of time for which an endpoint will cache a response, use its cache.time-to-live
property.
The following example sets the time-to-live of the beans
endpoint’s cache to 10 seconds:
management.endpoint.beans.cache.time-to-live=10s
management:
endpoint:
beans:
cache:
time-to-live: "10s"
The prefix management.endpoint.<name> is used to uniquely identify the endpoint that is being configured.
|
2.5. Hypermedia for Actuator Web Endpoints
A “discovery page” is added with links to all the endpoints.
The “discovery page” is available on /actuator
by default.
To disable the “discovery page”, add the following property to your application properties:
management.endpoints.web.discovery.enabled=false
management:
endpoints:
web:
discovery:
enabled: false
When a custom management context path is configured, the “discovery page” automatically moves from /actuator
to the root of the management context.
For example, if the management context path is /management
, then the discovery page is available from /management
.
When the management context path is set to /
, the discovery page is disabled to prevent the possibility of a clash with other mappings.
2.6. CORS Support
Cross-origin resource sharing (CORS) is a W3C specification that lets you specify in a flexible way what kind of cross-domain requests are authorized. If you use Spring MVC or Spring WebFlux, Actuator’s web endpoints can be configured to support such scenarios.
CORS support is disabled by default and is only enabled once the management.endpoints.web.cors.allowed-origins
property has been set.
The following configuration permits GET
and POST
calls from the example.com
domain:
management.endpoints.web.cors.allowed-origins=https://example.com
management.endpoints.web.cors.allowed-methods=GET,POST
management:
endpoints:
web:
cors:
allowed-origins: "https://example.com"
allowed-methods: "GET,POST"
See CorsEndpointProperties for a complete list of options. |
2.7. Implementing Custom Endpoints
If you add a @Bean
annotated with @Endpoint
, any methods annotated with @ReadOperation
, @WriteOperation
, or @DeleteOperation
are automatically exposed over JMX and, in a web application, over HTTP as well.
Endpoints can be exposed over HTTP using Jersey, Spring MVC, or Spring WebFlux.
If both Jersey and Spring MVC are available, Spring MVC will be used.
The following example exposes a read operation that returns a custom object:
@ReadOperation
public CustomData getData() {
return new CustomData("test", 5);
}
You can also write technology-specific endpoints by using @JmxEndpoint
or @WebEndpoint
.
These endpoints are restricted to their respective technologies.
For example, @WebEndpoint
is exposed only over HTTP and not over JMX.
You can write technology-specific extensions by using @EndpointWebExtension
and @EndpointJmxExtension
.
These annotations let you provide technology-specific operations to augment an existing endpoint.
Finally, if you need access to web-framework-specific functionality, you can implement Servlet or Spring @Controller
and @RestController
endpoints at the cost of them not being available over JMX or when using a different web framework.
2.7.1. Receiving Input
Operations on an endpoint receive input via their parameters.
When exposed via the web, the values for these parameters are taken from the URL’s query parameters and from the JSON request body.
When exposed via JMX, the parameters are mapped to the parameters of the MBean’s operations.
Parameters are required by default.
They can be made optional by annotating them with either @javax.annotation.Nullable
or @org.springframework.lang.Nullable
.
Each root property in the JSON request body can be mapped to a parameter of the endpoint. Consider the following JSON request body:
{
"name": "test",
"counter": 42
}
This can be used to invoke a write operation that takes String name
and int counter
parameters, as shown in the following example:
@WriteOperation
public void updateData(String name, int counter) {
// injects "test" and 42
}
Because endpoints are technology agnostic, only simple types can be specified in the method signature.
In particular declaring a single parameter with a CustomData type defining a name and counter properties is not supported.
|
To allow the input to be mapped to the operation method’s parameters, Java code implementing an endpoint should be compiled with -parameters , and Kotlin code implementing an endpoint should be compiled with -java-parameters .
This will happen automatically if you are using Spring Boot’s Gradle plugin or if you are using Maven and spring-boot-starter-parent .
|
Input Type Conversion
The parameters passed to endpoint operation methods are, if necessary, automatically converted to the required type.
Before calling an operation method, the input received via JMX or an HTTP request is converted to the required types using an instance of ApplicationConversionService
as well as any Converter
or GenericConverter
beans qualified with @EndpointConverter
.
2.7.2. Custom Web Endpoints
Operations on an @Endpoint
, @WebEndpoint
, or @EndpointWebExtension
are automatically exposed over HTTP using Jersey, Spring MVC, or Spring WebFlux.
If both Jersey and Spring MVC are available, Spring MVC will be used.
Web Endpoint Request Predicates
A request predicate is automatically generated for each operation on a web-exposed endpoint.
Path
The path of the predicate is determined by the ID of the endpoint and the base path of web-exposed endpoints.
The default base path is /actuator
.
For example, an endpoint with the ID sessions
will use /actuator/sessions
as its path in the predicate.
The path can be further customized by annotating one or more parameters of the operation method with @Selector
.
Such a parameter is added to the path predicate as a path variable.
The variable’s value is passed into the operation method when the endpoint operation is invoked.
If you want to capture all remaining path elements, you can add @Selector(Match=ALL_REMAINING)
to the last parameter and make it a type that is conversion compatible with a String[]
.
HTTP method
The HTTP method of the predicate is determined by the operation type, as shown in the following table:
Operation | HTTP method |
---|---|
|
|
|
|
|
|
Consumes
For a @WriteOperation
(HTTP POST
) that uses the request body, the consumes clause of the predicate is application/vnd.spring-boot.actuator.v2+json, application/json
.
For all other operations the consumes clause is empty.
Produces
The produces clause of the predicate can be determined by the produces
attribute of the @DeleteOperation
, @ReadOperation
, and @WriteOperation
annotations.
The attribute is optional.
If it is not used, the produces clause is determined automatically.
If the operation method returns void
or Void
the produces clause is empty.
If the operation method returns a org.springframework.core.io.Resource
, the produces clause is application/octet-stream
.
For all other operations the produces clause is application/vnd.spring-boot.actuator.v2+json, application/json
.
Web Endpoint Response Status
The default response status for an endpoint operation depends on the operation type (read, write, or delete) and what, if anything, the operation returns.
A @ReadOperation
returns a value, the response status will be 200 (OK).
If it does not return a value, the response status will be 404 (Not Found).
If a @WriteOperation
or @DeleteOperation
returns a value, the response status will be 200 (OK).
If it does not return a value the response status will be 204 (No Content).
If an operation is invoked without a required parameter, or with a parameter that cannot be converted to the required type, the operation method will not be called and the response status will be 400 (Bad Request).
Web Endpoint Range Requests
An HTTP range request can be used to request part of an HTTP resource.
When using Spring MVC or Spring Web Flux, operations that return a org.springframework.core.io.Resource
automatically support range requests.
Range requests are not supported when using Jersey. |
Web Endpoint Security
An operation on a web endpoint or a web-specific endpoint extension can receive the current java.security.Principal
or org.springframework.boot.actuate.endpoint.SecurityContext
as a method parameter.
The former is typically used in conjunction with @Nullable
to provide different behavior for authenticated and unauthenticated users.
The latter is typically used to perform authorization checks using its isUserInRole(String)
method.
2.7.3. Servlet Endpoints
A Servlet
can be exposed as an endpoint by implementing a class annotated with @ServletEndpoint
that also implements Supplier<EndpointServlet>
.
Servlet endpoints provide deeper integration with the Servlet container but at the expense of portability.
They are intended to be used to expose an existing Servlet
as an endpoint.
For new endpoints, the @Endpoint
and @WebEndpoint
annotations should be preferred whenever possible.
2.7.4. Controller Endpoints
@ControllerEndpoint
and @RestControllerEndpoint
can be used to implement an endpoint that is only exposed by Spring MVC or Spring WebFlux.
Methods are mapped using the standard annotations for Spring MVC and Spring WebFlux such as @RequestMapping
and @GetMapping
, with the endpoint’s ID being used as a prefix for the path.
Controller endpoints provide deeper integration with Spring’s web frameworks but at the expense of portability.
The @Endpoint
and @WebEndpoint
annotations should be preferred whenever possible.
2.8. Health Information
You can use health information to check the status of your running application.
It is often used by monitoring software to alert someone when a production system goes down.
The information exposed by the health
endpoint depends on the management.endpoint.health.show-details
and management.endpoint.health.show-components
properties which can be configured with one of the following values:
Name | Description |
---|---|
|
Details are never shown. |
|
Details are only shown to authorized users.
Authorized roles can be configured using |
|
Details are shown to all users. |
The default value is never
.
A user is considered to be authorized when they are in one or more of the endpoint’s roles.
If the endpoint has no configured roles (the default) all authenticated users are considered to be authorized.
The roles can be configured using the management.endpoint.health.roles
property.
If you have secured your application and wish to use always , your security configuration must permit access to the health endpoint for both authenticated and unauthenticated users.
|
Health information is collected from the content of a HealthContributorRegistry
(by default all HealthContributor
instances defined in your ApplicationContext
).
Spring Boot includes a number of auto-configured HealthContributors
and you can also write your own.
A HealthContributor
can either be a HealthIndicator
or a CompositeHealthContributor
.
A HealthIndicator
provides actual health information, including a Status
.
A CompositeHealthContributor
provides a composite of other HealthContributors
.
Taken together, contributors form a tree structure to represent the overall system health.
By default, the final system health is derived by a StatusAggregator
which sorts the statuses from each HealthIndicator
based on an ordered list of statuses.
The first status in the sorted list is used as the overall health status.
If no HealthIndicator
returns a status that is known to the StatusAggregator
, an UNKNOWN
status is used.
The HealthContributorRegistry can be used to register and unregister health indicators at runtime.
|
2.8.1. Auto-configured HealthIndicators
The following HealthIndicators
are auto-configured by Spring Boot when appropriate.
You can also enable/disable selected indicators by configuring management.health.key.enabled
,
with the key
listed in the table below.
Key | Name | Description |
---|---|---|
|
Checks that a Cassandra database is up. |
|
|
Checks that a Couchbase cluster is up. |
|
|
Checks that a connection to |
|
|
Checks for low disk space. |
|
|
Checks that an Elasticsearch cluster is up. |
|
|
Checks that a Hazelcast server is up. |
|
|
Checks that an InfluxDB server is up. |
|
|
Checks that a JMS broker is up. |
|
|
Checks that an LDAP server is up. |
|
|
Checks that a mail server is up. |
|
|
Checks that a Mongo database is up. |
|
|
Checks that a Neo4j database is up. |
|
|
Always responds with |
|
|
Checks that a Rabbit server is up. |
|
|
Checks that a Redis server is up. |
|
|
Checks that a Solr server is up. |
You can disable them all by setting the management.health.defaults.enabled property.
|
Additional HealthIndicators
are available but not enabled by default:
Key | Name | Description |
---|---|---|
|
Exposes the "Liveness" application availability state. |
|
|
Exposes the "Readiness" application availability state. |
2.8.2. Writing Custom HealthIndicators
To provide custom health information, you can register Spring beans that implement the HealthIndicator
interface.
You need to provide an implementation of the health()
method and return a Health
response.
The Health
response should include a status and can optionally include additional details to be displayed.
The following code shows a sample HealthIndicator
implementation:
@Component
public class MyHealthIndicator implements HealthIndicator {
@Override
public Health health() {
int errorCode = check();
if (errorCode != 0) {
return Health.down().withDetail("Error Code", errorCode).build();
}
return Health.up().build();
}
private int check() {
// perform some specific health check
return ...
}
}
The identifier for a given HealthIndicator is the name of the bean without the HealthIndicator suffix, if it exists.
In the preceding example, the health information is available in an entry named my .
|
In addition to Spring Boot’s predefined Status
types, it is also possible for Health
to return a custom Status
that represents a new system state.
In such cases, a custom implementation of the StatusAggregator
interface also needs to be provided, or the default implementation has to be configured by using the management.endpoint.health.status.order
configuration property.
For example, assume a new Status
with code FATAL
is being used in one of your HealthIndicator
implementations.
To configure the severity order, add the following property to your application properties:
management.endpoint.health.status.order=fatal,down,out-of-service,unknown,up
management:
endpoint:
health:
status:
order: "fatal,down,out-of-service,unknown,up"
The HTTP status code in the response reflects the overall health status.
By default, OUT_OF_SERVICE
and DOWN
map to 503.
Any unmapped health statuses, including UP
, map to 200.
You might also want to register custom status mappings if you access the health endpoint over HTTP.
Configuring a custom mapping disables the defaults mappings for DOWN
and OUT_OF_SERVICE
.
If you want to retain the default mappings they must be configured explicitly alongside any custom mappings.
For example, the following property maps FATAL
to 503 (service unavailable) and retains the default mappings for DOWN
and OUT_OF_SERVICE
:
management.endpoint.health.status.http-mapping.down=503
management.endpoint.health.status.http-mapping.fatal=503
management.endpoint.health.status.http-mapping.out-of-service=503
management:
endpoint:
health:
status:
http-mapping:
down: 503
fatal: 503
out-of-service: 503
If you need more control, you can define your own HttpCodeStatusMapper bean.
|
The following table shows the default status mappings for the built-in statuses:
Status | Mapping |
---|---|
|
|
|
|
|
No mapping by default, so HTTP status is |
|
No mapping by default, so HTTP status is |
2.8.3. Reactive Health Indicators
For reactive applications, such as those using Spring WebFlux, ReactiveHealthContributor
provides a non-blocking contract for getting application health.
Similar to a traditional HealthContributor
, health information is collected from the content of a ReactiveHealthContributorRegistry
(by default all HealthContributor
and ReactiveHealthContributor
instances defined in your ApplicationContext
).
Regular HealthContributors
that do not check against a reactive API are executed on the elastic scheduler.
In a reactive application, The ReactiveHealthContributorRegistry should be used to register and unregister health indicators at runtime.
If you need to register a regular HealthContributor , you should wrap it using ReactiveHealthContributor#adapt .
|
To provide custom health information from a reactive API, you can register Spring beans that implement the ReactiveHealthIndicator
interface.
The following code shows a sample ReactiveHealthIndicator
implementation:
@Component
public class MyReactiveHealthIndicator implements ReactiveHealthIndicator {
@Override
public Mono<Health> health() {
return doHealthCheck().onErrorResume((exception) ->
Mono.just(new Health.Builder().down(exception).build()));
}
private Mono<Health> doHealthCheck() {
// perform some specific health check
return ...
}
}
To handle the error automatically, consider extending from AbstractReactiveHealthIndicator .
|
2.8.4. Auto-configured ReactiveHealthIndicators
The following ReactiveHealthIndicators
are auto-configured by Spring Boot when appropriate:
Key | Name | Description |
---|---|---|
|
Checks that a Cassandra database is up. |
|
|
Checks that a Couchbase cluster is up. |
|
|
Checks that an Elasticsearch cluster is up. |
|
|
Checks that a Mongo database is up. |
|
|
Checks that a Neo4j database is up. |
|
|
Checks that a Redis server is up. |
If necessary, reactive indicators replace the regular ones.
Also, any HealthIndicator that is not handled explicitly is wrapped automatically.
|
2.8.5. Health Groups
It’s sometimes useful to organize health indicators into groups that can be used for different purposes.
To create a health indicator group you can use the management.endpoint.health.group.<name>
property and specify a list of health indicator IDs to include
or exclude
.
For example, to create a group that includes only database indicators you can define the following:
management.endpoint.health.group.custom.include=db
management:
endpoint:
health:
group:
custom:
include: "db"
You can then check the result by hitting localhost:8080/actuator/health/custom
.
Similarly, to create a group that excludes the database indicators from the group and includes all the other indicators, you can define the following:
management.endpoint.health.group.custom.exclude=db
management:
endpoint:
health:
group:
custom:
exclude: "db"
By default groups will inherit the same StatusAggregator
and HttpCodeStatusMapper
settings as the system health, however, these can also be defined on a per-group basis.
It’s also possible to override the show-details
and roles
properties if required:
management.endpoint.health.group.custom.show-details=when-authorized
management.endpoint.health.group.custom.roles=admin
management.endpoint.health.group.custom.status.order=fatal,up
management.endpoint.health.group.custom.status.http-mapping.fatal=500
management.endpoint.health.group.custom.status.http-mapping.out-of-service=500
management:
endpoint:
health:
group:
custom:
show-details: "when-authorized"
roles: "admin"
status:
order: "fatal,up"
http-mapping:
fatal: 500
out-of-service: 500
You can use @Qualifier("groupname") if you need to register custom StatusAggregator or HttpCodeStatusMapper beans for use with the group.
|
2.8.6. DataSource Health
The DataSource
health indicator shows the health of both standard data source and routing data source beans.
The health of a routing data source includes the health of each of its target data sources.
In the health endpoint’s response, each of a routing data source’s targets is named using its routing key.
If you prefer not to include routing data sources in the indicator’s output, set management.health.db.ignore-routing-data-sources
to true
.
2.9. Kubernetes Probes
Applications deployed on Kubernetes can provide information about their internal state with Container Probes. Depending on your Kubernetes configuration, the kubelet will call those probes and react to the result.
Spring Boot manages your Application Availability State out-of-the-box.
If deployed in a Kubernetes environment, actuator will gather the "Liveness" and "Readiness" information from the ApplicationAvailability
interface and use that information in dedicated Health Indicators: LivenessStateHealthIndicator
and ReadinessStateHealthIndicator
.
These indicators will be shown on the global health endpoint ("/actuator/health"
).
They will also be exposed as separate HTTP Probes using Health Groups: "/actuator/health/liveness"
and "/actuator/health/readiness"
.
You can then configure your Kubernetes infrastructure with the following endpoint information:
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: <actuator-port>
failureThreshold: ...
periodSeconds: ...
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: <actuator-port>
failureThreshold: ...
periodSeconds: ...
<actuator-port> should be set to the port that the actuator endpoints are available on.
It could be the main web server port, or a separate management port if the "management.server.port" property has been set.
|
These health groups are only enabled automatically if the application is running in a Kubernetes environment.
You can enable them in any environment using the management.endpoint.health.probes.enabled
configuration property.
If an application takes longer to start than the configured liveness period, Kubernetes mention the "startupProbe" as a possible solution.
The "startupProbe" is not necessarily needed here as the "readinessProbe" fails until all startup tasks are done, see how Probes behave during the application lifecycle.
|
If your Actuator endpoints are deployed on a separate management context, be aware that endpoints are then not using the same web infrastructure (port, connection pools, framework components) as the main application. In this case, a probe check could be successful even if the main application does not work properly (for example, it cannot accept new connections). |
2.9.1. Checking External State with Kubernetes Probes
Actuator configures the "liveness" and "readiness" probes as Health Groups; this means that all the Health Groups features are available for them. You can, for example, configure additional Health Indicators:
management.endpoint.health.group.readiness.include=readinessState,customCheck
management:
endpoint:
health:
group:
readiness:
include: "readinessState,customCheck"
By default, Spring Boot does not add other Health Indicators to these groups.
The “liveness” Probe should not depend on health checks for external systems. If the Liveness State of an application is broken, Kubernetes will try to solve that problem by restarting the application instance. This means that if an external system fails (e.g. a database, a Web API, an external cache), Kubernetes might restart all application instances and create cascading failures.
As for the “readiness” Probe, the choice of checking external systems must be made carefully by the application developers, i.e. Spring Boot does not include any additional health checks in the readiness probe. If the Readiness State of an application instance is unready, Kubernetes will not route traffic to that instance. Some external systems might not be shared by application instances, in which case they could quite naturally be included in a readiness probe. Other external systems might not be essential to the application (the application could have circuit breakers and fallbacks), in which case they definitely should not be included. Unfortunately, an external system that is shared by all application instances is common, and you have to make a judgement call: include it in the readiness probe and expect that the application is taken out of service when the external service is down, or leave it out and deal with failures higher up the stack, e.g. using a circuit breaker in the caller.
If all instances of an application are unready, a Kubernetes Service with type=ClusterIP or NodePort will not accept any incoming connections.
There is no HTTP error response (503 etc.) since there is no connection.
A Service with type=LoadBalancer might or might not accept connections, depending on the provider.
A Service that has an explicit Ingress will also respond in a way that depends on the implementation - the ingress service itself will have to decide how to handle the "connection refused" from downstream.
HTTP 503 is quite likely in the case of both load balancer and ingress.
|
Also, if an application is using Kubernetes autoscaling it may react differently to applications being taken out of the load-balancer, depending on its autoscaler configuration.
2.9.2. Application Lifecycle and Probe States
An important aspect of the Kubernetes Probes support is its consistency with the application lifecycle.
There is a significant difference between the AvailabilityState
which is the in-memory, internal state of the application
and the actual Probe which exposes that state: depending on the phase of application lifecycle, the Probe might not be available.
Spring Boot publishes Application Events during startup and shutdown,
and Probes can listen to such events and expose the AvailabilityState
information.
The following tables show the AvailabilityState
and the state of HTTP connectors at different stages.
When a Spring Boot application starts:
Startup phase | LivenessState | ReadinessState | HTTP server | Notes |
---|---|---|---|---|
Starting |
|
|
Not started |
Kubernetes checks the "liveness" Probe and restarts the application if it takes too long. |
Started |
|
|
Refuses requests |
The application context is refreshed. The application performs startup tasks and does not receive traffic yet. |
Ready |
|
|
Accepts requests |
Startup tasks are finished. The application is receiving traffic. |
When a Spring Boot application shuts down:
Shutdown phase | Liveness State | Readiness State | HTTP server | Notes |
---|---|---|---|---|
Running |
|
|
Accepts requests |
Shutdown has been requested. |
Graceful shutdown |
|
|
New requests are rejected |
If enabled, graceful shutdown processes in-flight requests. |
Shutdown complete |
N/A |
N/A |
Server is shut down |
The application context is closed and the application is shut down. |
Check out the Kubernetes container lifecycle section for more information about Kubernetes deployment. |
2.10. Application Information
Application information exposes various information collected from all InfoContributor
beans defined in your ApplicationContext
.
Spring Boot includes a number of auto-configured InfoContributor
beans, and you can write your own.
2.10.1. Auto-configured InfoContributors
The following InfoContributor
beans are auto-configured by Spring Boot, when appropriate:
Name | Description |
---|---|
Exposes any key from the |
|
Exposes git information if a |
|
Exposes build information if a |
It is possible to disable them all by setting the management.info.defaults.enabled property.
|
2.10.2. Custom Application Information
You can customize the data exposed by the info
endpoint by setting info.*
Spring properties.
All Environment
properties under the info
key are automatically exposed.
For example, you could add the following settings to your application.properties
file:
info.app.encoding=UTF-8
info.app.java.source=11
info.app.java.target=11
info:
app:
encoding: "UTF-8"
java:
source: "11"
target: "11"
Rather than hardcoding those values, you could also expand info properties at build time. Assuming you use Maven, you could rewrite the preceding example as follows: Properties
Yaml
|
2.10.3. Git Commit Information
Another useful feature of the info
endpoint is its ability to publish information about the state of your git
source code repository when the project was built.
If a GitProperties
bean is available, the info
endpoint can be used to expose these properties.
A GitProperties bean is auto-configured if a git.properties file is available at the root of the classpath.
See "Generate git information" for more details.
|
By default, the endpoint exposes git.branch
, git.commit.id
, and git.commit.time
properties, if present.
If you don’t want any of these properties in the endpoint response, they need to be excluded from the git.properties
file.
If you want to display the full git information (that is, the full content of git.properties
), use the management.info.git.mode
property, as follows:
management.info.git.mode=full
management:
info:
git:
mode: "full"
To disable the git commit information from the info
endpoint completely, set the management.info.git.enabled
property to false
, as follows:
management.info.git.enabled=false
management:
info:
git:
enabled: false
2.10.4. Build Information
If a BuildProperties
bean is available, the info
endpoint can also publish information about your build.
This happens if a META-INF/build-info.properties
file is available in the classpath.
The Maven and Gradle plugins can both generate that file. See "Generate build information" for more details. |
2.10.5. Writing Custom InfoContributors
To provide custom application information, you can register Spring beans that implement the InfoContributor
interface.
The following example contributes an example
entry with a single value:
@Component
public class MyInfoContributor implements InfoContributor {
@Override
public void contribute(Info.Builder builder) {
builder.withDetail("example", Collections.singletonMap("key", "value"));
}
}
If you reach the info
endpoint, you should see a response that contains the following additional entry:
{
"example": {
"key" : "value"
}
}
3. Monitoring and Management over HTTP
If you are developing a web application, Spring Boot Actuator auto-configures all enabled endpoints to be exposed over HTTP.
The default convention is to use the id
of the endpoint with a prefix of /actuator
as the URL path.
For example, health
is exposed as /actuator/health
.
Actuator is supported natively with Spring MVC, Spring WebFlux, and Jersey. If both Jersey and Spring MVC are available, Spring MVC will be used. |
Jackson is a required dependency in order to get the correct JSON responses as documented in the API documentation (HTML or PDF). |
3.1. Customizing the Management Endpoint Paths
Sometimes, it is useful to customize the prefix for the management endpoints.
For example, your application might already use /actuator
for another purpose.
You can use the management.endpoints.web.base-path
property to change the prefix for your management endpoint, as shown in the following example:
management.endpoints.web.base-path=/manage
management:
endpoints:
web:
base-path: "/manage"
The preceding application.properties
example changes the endpoint from /actuator/{id}
to /manage/{id}
(for example, /manage/info
).
Unless the management port has been configured to expose endpoints by using a different HTTP port, management.endpoints.web.base-path is relative to server.servlet.context-path (Servlet web applications) or spring.webflux.base-path (reactive web applications).
If management.server.port is configured, management.endpoints.web.base-path is relative to management.server.base-path .
|
If you want to map endpoints to a different path, you can use the management.endpoints.web.path-mapping
property.
The following example remaps /actuator/health
to /healthcheck
:
management.endpoints.web.base-path=/
management.endpoints.web.path-mapping.health=healthcheck
management:
endpoints:
web:
base-path: "/"
path-mapping:
health: "healthcheck"
3.2. Customizing the Management Server Port
Exposing management endpoints by using the default HTTP port is a sensible choice for cloud-based deployments. If, however, your application runs inside your own data center, you may prefer to expose endpoints by using a different HTTP port.
You can set the management.server.port
property to change the HTTP port, as shown in the following example:
management.server.port=8081
management:
server:
port: 8081
On Cloud Foundry, applications only receive requests on port 8080 for both HTTP and TCP routing, by default. If you want to use a custom management port on Cloud Foundry, you will need to explicitly set up the application’s routes to forward traffic to the custom port. |
3.3. Configuring Management-specific SSL
When configured to use a custom port, the management server can also be configured with its own SSL by using the various management.server.ssl.*
properties.
For example, doing so lets a management server be available over HTTP while the main application uses HTTPS, as shown in the following property settings:
server.port=8443
server.ssl.enabled=true
server.ssl.key-store=classpath:store.jks
server.ssl.key-password=secret
management.server.port=8080
management.server.ssl.enabled=false
server:
port: 8443
ssl:
enabled: true
key-store: "classpath:store.jks"
key-password: secret
management:
server:
port: 8080
ssl:
enabled: false
Alternatively, both the main server and the management server can use SSL but with different key stores, as follows:
server.port=8443
server.ssl.enabled=true
server.ssl.key-store=classpath:main.jks
server.ssl.key-password=secret
management.server.port=8080
management.server.ssl.enabled=true
management.server.ssl.key-store=classpath:management.jks
management.server.ssl.key-password=secret
server:
port: 8443
ssl:
enabled: true
key-store: "classpath:main.jks"
key-password: "secret"
management:
server:
port: 8080
ssl:
enabled: true
key-store: "classpath:management.jks"
key-password: "secret"
3.4. Customizing the Management Server Address
You can customize the address that the management endpoints are available on by setting the management.server.address
property.
Doing so can be useful if you want to listen only on an internal or ops-facing network or to listen only for connections from localhost
.
You can listen on a different address only when the port differs from the main server port. |
The following example application.properties
does not allow remote management connections:
management.server.port=8081
management.server.address=127.0.0.1
management:
server:
port: 8081
address: "127.0.0.1"
3.5. Disabling HTTP Endpoints
If you do not want to expose endpoints over HTTP, you can set the management port to -1
, as shown in the following example:
management.server.port=-1
management:
server:
port: -1
This can be achieved using the management.endpoints.web.exposure.exclude
property as well, as shown in the following example:
management.endpoints.web.exposure.exclude=*
management:
endpoints:
web:
exposure:
exclude: "*"
4. Monitoring and Management over JMX
Java Management Extensions (JMX) provide a standard mechanism to monitor and manage applications.
By default, this feature is not enabled and can be turned on by setting the configuration property spring.jmx.enabled
to true
.
Spring Boot exposes management endpoints as JMX MBeans under the org.springframework.boot
domain by default.
To Take full control over endpoints registration in the JMX domain, consider registering your own EndpointObjectNameFactory
implementation.
4.1. Customizing MBean Names
The name of the MBean is usually generated from the id
of the endpoint.
For example, the health
endpoint is exposed as org.springframework.boot:type=Endpoint,name=Health
.
If your application contains more than one Spring ApplicationContext
, you may find that names clash.
To solve this problem, you can set the spring.jmx.unique-names
property to true
so that MBean names are always unique.
You can also customize the JMX domain under which endpoints are exposed.
The following settings show an example of doing so in application.properties
:
spring.jmx.unique-names=true
management.endpoints.jmx.domain=com.example.myapp
spring:
jmx:
unique-names: true
management:
endpoints:
jmx:
domain: "com.example.myapp"
4.2. Disabling JMX Endpoints
If you do not want to expose endpoints over JMX, you can set the management.endpoints.jmx.exposure.exclude
property to *
, as shown in the following example:
management.endpoints.jmx.exposure.exclude=*
management:
endpoints:
jmx:
exposure:
exclude: "*"
4.3. Using Jolokia for JMX over HTTP
Jolokia is a JMX-HTTP bridge that provides an alternative method of accessing JMX beans.
To use Jolokia, include a dependency to org.jolokia:jolokia-core
.
For example, with Maven, you would add the following dependency:
<dependency>
<groupId>org.jolokia</groupId>
<artifactId>jolokia-core</artifactId>
</dependency>
The Jolokia endpoint can then be exposed by adding jolokia
or *
to the management.endpoints.web.exposure.include
property.
You can then access it by using /actuator/jolokia
on your management HTTP server.
The Jolokia endpoint exposes Jolokia’s servlet as an actuator endpoint. As a result, it is specific to servlet environments such as Spring MVC and Jersey. The endpoint will not be available in a WebFlux application. |
4.3.1. Customizing Jolokia
Jolokia has a number of settings that you would traditionally configure by setting servlet parameters.
With Spring Boot, you can use your application.properties
file.
To do so, prefix the parameter with management.endpoint.jolokia.config.
, as shown in the following example:
management.endpoint.jolokia.config.debug=true
management:
endpoint:
jolokia:
config:
debug: true
5. Loggers
Spring Boot Actuator includes the ability to view and configure the log levels of your application at runtime. You can view either the entire list or an individual logger’s configuration, which is made up of both the explicitly configured logging level as well as the effective logging level given to it by the logging framework. These levels can be one of:
-
TRACE
-
DEBUG
-
INFO
-
WARN
-
ERROR
-
FATAL
-
OFF
-
null
null
indicates that there is no explicit configuration.
5.1. Configure a Logger
To configure a given logger, POST
a partial entity to the resource’s URI, as shown in the following example:
{
"configuredLevel": "DEBUG"
}
To “reset” the specific level of the logger (and use the default configuration instead), you can pass a value of null as the configuredLevel .
|
6. Metrics
Spring Boot Actuator provides dependency management and auto-configuration for Micrometer, an application metrics facade that supports numerous monitoring systems, including:
To learn more about Micrometer’s capabilities, please refer to its reference documentation, in particular the concepts section. |
6.1. Getting started
Spring Boot auto-configures a composite MeterRegistry
and adds a registry to the composite for each of the supported implementations that it finds on the classpath.
Having a dependency on micrometer-registry-{system}
in your runtime classpath is enough for Spring Boot to configure the registry.
Most registries share common features. For instance, you can disable a particular registry even if the Micrometer registry implementation is on the classpath. For example, to disable Datadog:
management.metrics.export.datadog.enabled=false
management:
metrics:
export:
datadog:
enabled: false
You can also disable all registries unless stated otherwise by the registry-specific property, as shown in the following example:
management.metrics.export.defaults.enabled=false
management:
metrics:
export:
defaults:
enabled: false
Spring Boot will also add any auto-configured registries to the global static composite registry on the Metrics
class unless you explicitly tell it not to:
management.metrics.use-global-registry=false
management:
metrics:
use-global-registry: false
You can register any number of MeterRegistryCustomizer
beans to further configure the registry, such as applying common tags, before any meters are registered with the registry:
@Configuration(proxyBeanMethods = false)
public class MyMeterRegistryConfiguration {
@Bean
public MeterRegistryCustomizer<MeterRegistry> metricsCommonTags() {
return (registry) -> registry.config().commonTags("region", "us-east-1");
}
}
You can apply customizations to particular registry implementations by being more specific about the generic type:
@Configuration(proxyBeanMethods = false)
public class MyMeterRegistryConfiguration {
@Bean
public MeterRegistryCustomizer<GraphiteMeterRegistry> graphiteMetricsNamingConvention() {
return (registry) -> registry.config().namingConvention(this::name);
}
private String name(String name, Meter.Type type, String baseUnit) {
return ...
}
}
Spring Boot also configures built-in instrumentation that you can control via configuration or dedicated annotation markers.
6.2. Supported Monitoring Systems
6.2.1. AppOptics
By default, the AppOptics registry pushes metrics to api.appoptics.com/v1/measurements
periodically.
To export metrics to SaaS AppOptics, your API token must be provided:
management.metrics.export.appoptics.api-token=YOUR_TOKEN
management:
metrics:
export:
appoptics:
api-token: "YOUR_TOKEN"
6.2.2. Atlas
By default, metrics are exported to Atlas running on your local machine. The location of the Atlas server to use can be provided using:
management.metrics.export.atlas.uri=https://atlas.example.com:7101/api/v1/publish
management:
metrics:
export:
atlas:
uri: "https://atlas.example.com:7101/api/v1/publish"
6.2.3. Datadog
Datadog registry pushes metrics to datadoghq periodically. To export metrics to Datadog, your API key must be provided:
management.metrics.export.datadog.api-key=YOUR_KEY
management:
metrics:
export:
datadog:
api-key: "YOUR_KEY"
You can also change the interval at which metrics are sent to Datadog:
management.metrics.export.datadog.step=30s
management:
metrics:
export:
datadog:
step: "30s"
6.2.4. Dynatrace
Dynatrace registry pushes metrics to the configured URI periodically. To export metrics to Dynatrace, your API token, device ID, and URI must be provided:
management.metrics.export.dynatrace.api-token=YOUR_TOKEN
management.metrics.export.dynatrace.device-id=YOUR_DEVICE_ID
management.metrics.export.dynatrace.uri=YOUR_URI
management:
metrics:
export:
dynatrace:
api-token: "YOUR_TOKEN"
device-id: "YOUR_DEVICE_ID"
uri: "YOUR_URI"
You can also change the interval at which metrics are sent to Dynatrace:
management.metrics.export.dynatrace.step=30s
management:
metrics:
export:
dynatrace:
step: "30s"
6.2.5. Elastic
By default, metrics are exported to Elastic running on your local machine. The location of the Elastic server to use can be provided using the following property:
management.metrics.export.elastic.host=https://elastic.example.com:8086
management:
metrics:
export:
elastic:
host: "https://elastic.example.com:8086"
6.2.6. Ganglia
By default, metrics are exported to Ganglia running on your local machine. The Ganglia server host and port to use can be provided using:
management.metrics.export.ganglia.host=ganglia.example.com
management.metrics.export.ganglia.port=9649
management:
metrics:
export:
ganglia:
host: "ganglia.example.com"
port: 9649
6.2.7. Graphite
By default, metrics are exported to Graphite running on your local machine. The Graphite server host and port to use can be provided using:
management.metrics.export.graphite.host=graphite.example.com
management.metrics.export.graphite.port=9004
management:
metrics:
export:
graphite:
host: "graphite.example.com"
port: 9004
Micrometer provides a default HierarchicalNameMapper
that governs how a dimensional meter id is mapped to flat hierarchical names.
To take control over this behavior, define your GraphiteMeterRegistry and supply your own HierarchicalNameMapper .
An auto-configured GraphiteConfig and Clock beans are provided unless you define your own:
|
@Configuration(proxyBeanMethods = false)
public class MyGraphiteConfiguration {
@Bean
public GraphiteMeterRegistry graphiteMeterRegistry(GraphiteConfig config, Clock clock) {
return new GraphiteMeterRegistry(config, clock, this::toHierarchicalName);
}
private String toHierarchicalName(Meter.Id id, NamingConvention convention) {
return ...
}
}
6.2.8. Humio
By default, the Humio registry pushes metrics to cloud.humio.com periodically. To export metrics to SaaS Humio, your API token must be provided:
management.metrics.export.humio.api-token=YOUR_TOKEN
management:
metrics:
export:
humio:
api-token: "YOUR_TOKEN"
You should also configure one or more tags to identify the data source to which metrics will be pushed:
management.metrics.export.humio.tags.alpha=a
management.metrics.export.humio.tags.bravo=b
management:
metrics:
export:
humio:
tags:
alpha: "a"
bravo: "b"
6.2.9. Influx
By default, metrics are exported to an Influx v1 instance running on your local machine with the default configuration.
To export metrics to InfluxDB v2, configure the org
, bucket
, and authentication token
for writing metrics.
The location of the Influx server to use can be provided using:
management.metrics.export.influx.uri=https://influx.example.com:8086
management:
metrics:
export:
influx:
uri: "https://influx.example.com:8086"
6.2.10. JMX
Micrometer provides a hierarchical mapping to JMX, primarily as a cheap and portable way to view metrics locally.
By default, metrics are exported to the metrics
JMX domain.
The domain to use can be provided using:
management.metrics.export.jmx.domain=com.example.app.metrics
management:
metrics:
export:
jmx:
domain: "com.example.app.metrics"
Micrometer provides a default HierarchicalNameMapper
that governs how a dimensional meter id is mapped to flat hierarchical names.
To take control over this behavior, define your JmxMeterRegistry and supply your own HierarchicalNameMapper .
An auto-configured JmxConfig and Clock beans are provided unless you define your own:
|
@Configuration(proxyBeanMethods = false)
public class MyJmxConfiguration {
@Bean
public JmxMeterRegistry jmxMeterRegistry(JmxConfig config, Clock clock) {
return new JmxMeterRegistry(config, clock, this::toHierarchicalName);
}
private String toHierarchicalName(Meter.Id id, NamingConvention convention) {
return ...
}
}
6.2.11. KairosDB
By default, metrics are exported to KairosDB running on your local machine. The location of the KairosDB server to use can be provided using:
management.metrics.export.kairos.uri=https://kairosdb.example.com:8080/api/v1/datapoints
management:
metrics:
export:
kairos:
uri: "https://kairosdb.example.com:8080/api/v1/datapoints"
6.2.12. New Relic
New Relic registry pushes metrics to New Relic periodically. To export metrics to New Relic, your API key and account id must be provided:
management.metrics.export.newrelic.api-key=YOUR_KEY
management.metrics.export.newrelic.account-id=YOUR_ACCOUNT_ID
management:
metrics:
export:
newrelic:
api-key: "YOUR_KEY"
account-id: "YOUR_ACCOUNT_ID"
You can also change the interval at which metrics are sent to New Relic:
management.metrics.export.newrelic.step=30s
management:
metrics:
export:
newrelic:
step: "30s"
By default, metrics are published via REST calls but it is also possible to use the Java Agent API if you have it on the classpath:
management.metrics.export.newrelic.client-provider-type=insights-agent
management:
metrics:
export:
newrelic:
client-provider-type: "insights-agent"
Finally, you can take full control by defining your own NewRelicClientProvider
bean.
6.2.13. Prometheus
Prometheus expects to scrape or poll individual app instances for metrics.
Spring Boot provides an actuator endpoint available at /actuator/prometheus
to present a Prometheus scrape with the appropriate format.
The endpoint is not available by default and must be exposed, see exposing endpoints for more details. |
Here is an example scrape_config
to add to prometheus.yml
:
scrape_configs:
- job_name: 'spring'
metrics_path: '/actuator/prometheus'
static_configs:
- targets: ['HOST:PORT']
For ephemeral or batch jobs which may not exist long enough to be scraped, Prometheus Pushgateway support can be used to expose their metrics to Prometheus. To enable Prometheus Pushgateway support, add the following dependency to your project:
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_pushgateway</artifactId>
</dependency>
When the Prometheus Pushgateway dependency is present on the classpath and the management.metrics.export.prometheus.pushgateway.enabled
property is set to true
, a PrometheusPushGatewayManager
bean is auto-configured.
This manages the pushing of metrics to a Prometheus Pushgateway.
The PrometheusPushGatewayManager
can be tuned using properties under management.metrics.export.prometheus.pushgateway
.
For advanced configuration, you can also provide your own PrometheusPushGatewayManager
bean.
6.2.14. SignalFx
SignalFx registry pushes metrics to SignalFx periodically. To export metrics to SignalFx, your access token must be provided:
management.metrics.export.signalfx.access-token=YOUR_ACCESS_TOKEN
management:
metrics:
export:
signalfx:
access-token: "YOUR_ACCESS_TOKEN"
You can also change the interval at which metrics are sent to SignalFx:
management.metrics.export.signalfx.step=30s
management:
metrics:
export:
signalfx:
step: "30s"
6.2.15. Simple
Micrometer ships with a simple, in-memory backend that is automatically used as a fallback if no other registry is configured. This allows you to see what metrics are collected in the metrics endpoint.
The in-memory backend disables itself as soon as you’re using any of the other available backend. You can also disable it explicitly:
management.metrics.export.simple.enabled=false
management:
metrics:
export:
simple:
enabled: false
6.2.16. Stackdriver
Stackdriver registry pushes metrics to Stackdriver periodically. To export metrics to SaaS Stackdriver, your Google Cloud project id must be provided:
management.metrics.export.stackdriver.project-id=my-project
management:
metrics:
export:
stackdriver:
project-id: "my-project"
You can also change the interval at which metrics are sent to Stackdriver:
management.metrics.export.stackdriver.step=30s
management:
metrics:
export:
stackdriver:
step: "30s"
6.2.17. StatsD
The StatsD registry pushes metrics over UDP to a StatsD agent eagerly. By default, metrics are exported to a StatsD agent running on your local machine. The StatsD agent host, port, and protocol to use can be provided using:
management.metrics.export.statsd.host=statsd.example.com
management.metrics.export.statsd.port=9125
management.metrics.export.statsd.protocol=udp
management:
metrics:
export:
statsd:
host: "statsd.example.com"
port: 9125
protocol: "udp"
You can also change the StatsD line protocol to use (default to Datadog):
management.metrics.export.statsd.flavor=etsy
management:
metrics:
export:
statsd:
flavor: "etsy"
6.2.18. Wavefront
Wavefront registry pushes metrics to Wavefront periodically. If you are exporting metrics to Wavefront directly, your API token must be provided:
management.metrics.export.wavefront.api-token=YOUR_API_TOKEN
management:
metrics:
export:
wavefront:
api-token: "YOUR_API_TOKEN"
Alternatively, you may use a Wavefront sidecar or an internal proxy set up in your environment that forwards metrics data to the Wavefront API host:
management.metrics.export.wavefront.uri=proxy://localhost:2878
management:
metrics:
export:
wavefront:
uri: "proxy://localhost:2878"
If publishing metrics to a Wavefront proxy (as described in the documentation), the host must be in the proxy://HOST:PORT format.
|
You can also change the interval at which metrics are sent to Wavefront:
management.metrics.export.wavefront.step=30s
management:
metrics:
export:
wavefront:
step: "30s"
6.3. Supported Metrics and Meters
Spring Boot provides automatic meter registration for a wide variety of technologies. In most situations, the out-of-the-box defaults will provide sensible metrics that can be published to any of the supported monioring systems.
6.3.1. JVM Metrics
Auto-configuration will enable JVM Metrics using core Micrometer classes.
JVM metrics are published under the jvm.
meter name.
The following JVM metrics are provided:
-
Various memory and buffer pool details
-
Statistics related to garbage collection
-
Threads utilization
-
The Number of classes loaded/unloaded
6.3.2. System Metrics
Auto-configuration will enable system metrics using core Micrometer classes.
System metrics are published under the system.
and process.
meter names.
The following system metrics are provided:
-
CPU metrics
-
File descriptor metrics
-
Uptime metrics (both the amount of time the application has been running as well as a fixed gauge of the absolute start time)
6.3.3. Logger Metrics
Auto-configuration enables the event metrics for both Logback and Log4J2.
Details are published under the log4j2.events.
or logback.events.
meter names.
6.3.4. Spring MVC Metrics
Auto-configuration enables the instrumentation of all requests handled by Spring MVC controllers and functional handlers.
By default, metrics are generated with the name, http.server.requests
.
The name can be customized by setting the management.metrics.web.server.request.metric-name
property.
@Timed
annotations are supported on @Controller
classes and @RequestMapping
methods (see @Timed Annotation Support for details).
If you don’t want to record metrics for all Spring MVC requests, you can set management.metrics.web.server.request.autotime.enabled
to false
and exclusively use @Timed
annotations instead.
By default, Spring MVC related metrics are tagged with the following information:
Tag | Description |
---|---|
|
Simple class name of any exception that was thrown while handling the request. |
|
Request’s method (for example, |
|
Request’s outcome based on the status code of the response.
1xx is |
|
Response’s HTTP status code (for example, |
|
Request’s URI template prior to variable substitution, if possible (for example, |
To add to the default tags, provide one or more @Bean
s that implement WebMvcTagsContributor
.
To replace the default tags, provide a @Bean
that implements WebMvcTagsProvider
.
In some cases, exceptions handled in Web controllers are not recorded as request metrics tags. Applications can opt-in and record exceptions by setting handled exceptions as request attributes. |
6.3.5. Spring WebFlux Metrics
Auto-configuration enables the instrumentation of all requests handled by Spring WebFlux controllers and functional handlers.
By default, metrics are generated with the name, http.server.requests
.
The name can be customized by setting the management.metrics.web.server.request.metric-name
property.
@Timed
annotations are supported on @Controller
classes and @RequestMapping
methods (see @Timed Annotation Support for details).
If you don’t want to record metrics for all Spring WebFlux requests, you can set management.metrics.web.server.request.autotime.enabled
to false
and exclusively use @Timed
annotations instead.
By default, WebFlux related metrics are tagged with the following information:
Tag | Description |
---|---|
|
Simple class name of any exception that was thrown while handling the request. |
|
Request’s method (for example, |
|
Request’s outcome based on the status code of the response.
1xx is |
|
Response’s HTTP status code (for example, |
|
Request’s URI template prior to variable substitution, if possible (for example, |
To add to the default tags, provide one or more @Bean
s that implement WebFluxTagsContributor
.
To replace the default tags, provide a @Bean
that implements WebFluxTagsProvider
.
In some cases, exceptions handled in controllers and handler functions are not recorded as request metrics tags. Applications can opt-in and record exceptions by setting handled exceptions as request attributes. |
6.3.6. Jersey Server Metrics
Auto-configuration enables the instrumentation of all requests handled by the Jersey JAX-RS implementation whenever Micrometer’s micrometer-jersey2
module is on the classpath.
By default, metrics are generated with the name, http.server.requests
.
The name can be customized by setting the management.metrics.web.server.request.metric-name
property.
@Timed
annotations are supported on request-handling classes and methods (see @Timed Annotation Support for details).
If you don’t want to record metrics for all Jersey requests, you can set management.metrics.web.server.request.autotime.enabled
to false
and exclusively use @Timed
annotations instead.
By default, Jersey server metrics are tagged with the following information:
Tag | Description |
---|---|
|
Simple class name of any exception that was thrown while handling the request. |
|
Request’s method (for example, |
|
Request’s outcome based on the status code of the response.
1xx is |
|
Response’s HTTP status code (for example, |
|
Request’s URI template prior to variable substitution, if possible (for example, |
To customize the tags, provide a @Bean
that implements JerseyTagsProvider
.
6.3.7. HTTP Client Metrics
Spring Boot Actuator manages the instrumentation of both RestTemplate
and WebClient
.
For that, you have to inject the auto-configured builder and use it to create instances:
-
RestTemplateBuilder
forRestTemplate
-
WebClient.Builder
forWebClient
It is also possible to apply manually the customizers responsible for this instrumentation, namely MetricsRestTemplateCustomizer
and MetricsWebClientCustomizer
.
By default, metrics are generated with the name, http.client.requests
.
The name can be customized by setting the management.metrics.web.client.request.metric-name
property.
By default, metrics generated by an instrumented client are tagged with the following information:
Tag | Description |
---|---|
|
Host portion of the URI |
|
Request’s method (for example, |
|
Request’s outcome based on the status code of the response.
1xx is |
|
Response’s HTTP status code if available (for example, |
|
Request’s URI template prior to variable substitution, if possible (for example, |
To customize the tags, and depending on your choice of client, you can provide a @Bean
that implements RestTemplateExchangeTagsProvider
or WebClientExchangeTagsProvider
.
There are convenience static functions in RestTemplateExchangeTags
and WebClientExchangeTags
.
6.3.8. Tomcat Metrics
Auto-configuration will enable the instrumentation of Tomcat only when an MBeanRegistry
is enabled.
By default, the MBeanRegistry
is disabled, but you can enable it by setting server.tomcat.mbeanregistry.enabled
to true
.
Tomcat metrics are published under the tomcat.
meter name.
6.3.9. Cache Metrics
Auto-configuration enables the instrumentation of all available Cache
s on startup with metrics prefixed with cache
.
Cache instrumentation is standardized for a basic set of metrics.
Additional, cache-specific metrics are also available.
The following cache libraries are supported:
-
Caffeine
-
EhCache 2
-
Hazelcast
-
Any compliant JCache (JSR-107) implementation
-
Redis
Metrics are tagged by the name of the cache and by the name of the CacheManager
that is derived from the bean name.
Only caches that are configured on startup are bound to the registry.
For caches not defined in the cache’s configuration, e.g. caches created on-the-fly or programmatically after the startup phase, an explicit registration is required.
A CacheMetricsRegistrar bean is made available to make that process easier.
|
6.3.10. DataSource Metrics
Auto-configuration enables the instrumentation of all available DataSource
objects with metrics prefixed with jdbc.connections
.
Data source instrumentation results in gauges representing the currently active, idle, maximum allowed, and minimum allowed connections in the pool.
Metrics are also tagged by the name of the DataSource
computed based on the bean name.
By default, Spring Boot provides metadata for all supported data sources; you can add additional DataSourcePoolMetadataProvider beans if your favorite data source isn’t supported out of the box.
See DataSourcePoolMetadataProvidersConfiguration for examples.
|
Also, Hikari-specific metrics are exposed with a hikaricp
prefix.
Each metric is tagged by the name of the Pool (can be controlled with spring.datasource.name
).
6.3.11. Hibernate Metrics
If org.hibernate:hibernate-micrometer
is on the classpath, all available Hibernate EntityManagerFactory
instances that have statistics enabled are instrumented with a metric named hibernate
.
Metrics are also tagged by the name of the EntityManagerFactory
that is derived from the bean name.
To enable statistics, the standard JPA property hibernate.generate_statistics
must be set to true
.
You can enable that on the auto-configured EntityManagerFactory
as shown in the following example:
spring.jpa.properties[hibernate.generate_statistics]=true
spring:
jpa:
properties:
"[hibernate.generate_statistics]": true
6.3.12. Spring Data Repository Metrics
Auto-configuration enables the instrumentation of all Spring Data Repository
method invocations.
By default, metrics are generated with the name, spring.data.repository.invocations
.
The name can be customized by setting the management.metrics.data.repository.metric-name
property.
@Timed
annotations are supported on Repository
classes and methods (see @Timed Annotation Support for details).
If you don’t want to record metrics for all Repository
invocations, you can set management.metrics.data.repository.autotime.enabled
to false
and exclusively use @Timed
annotations instead.
By default, repository invocation related metrics are tagged with the following information:
Tag | Description |
---|---|
|
Simple class name of the source |
|
The name of the |
|
The result state ( |
|
Simple class name of any exception that was thrown from the invocation. |
To replace the default tags, provide a @Bean
that implements RepositoryTagsProvider
.
6.3.13. RabbitMQ Metrics
Auto-configuration will enable the instrumentation of all available RabbitMQ connection factories with a metric named rabbitmq
.
6.3.14. Spring Integration Metrics
Spring Integration provides Micrometer support automatically whenever a MeterRegistry
bean is available.
Metrics are published under the spring.integration.
meter name.
6.3.15. Kafka Metrics
Auto-configuration will register a MicrometerConsumerListener
and MicrometerProducerListener
for the auto-configured consumer factory and producer factory respectively.
It will also register a KafkaStreamsMicrometerListener
for StreamsBuilderFactoryBean
.
For more details refer to Micrometer Native Metrics section of the Spring Kafka documentation.
6.3.16. MongoDB Metrics
Command Metrics
Auto-configuration will register a MongoMetricsCommandListener
with the auto-configured MongoClient
.
A timer metric with the name mongodb.driver.commands
is created for each command issued to the underlying MongoDB driver.
Each metric is tagged with the following information by default:
Tag | Description |
---|---|
|
Name of the command issued |
|
Identifier of the cluster the command was sent to |
|
Address of the server the command was sent to |
|
Outcome of the command - one of ( |
To replace the default metric tags, define a MongoCommandTagsProvider
bean, as shown in the following example:
@Configuration(proxyBeanMethods = false)
public class MyCommandTagsProviderConfiguration {
@Bean
public MongoCommandTagsProvider customCommandTagsProvider() {
return new CustomCommandTagsProvider();
}
}
To disable the auto-configured command metrics, set the following property:
management.metrics.mongo.command.enabled=false
management:
metrics:
mongo:
command:
enabled: false
Connection Pool Metrics
Auto-configuration will register a MongoMetricsConnectionPoolListener
with the auto-configured MongoClient
.
The following gauge metrics are created for the connection pool:
-
mongodb.driver.pool.size
that reports the current size of the connection pool, including idle and and in-use members -
mongodb.driver.pool.checkedout
that reports the count of connections that are currently in use -
mongodb.driver.pool.waitqueuesize
that reports the current size of the wait queue for a connection from the pool
Each metric is tagged with the following information by default:
Tag | Description |
---|---|
|
Identifier of the cluster the connection pool corresponds to |
|
Address of the server the connection pool corresponds to |
To replace the default metric tags, define a MongoConnectionPoolTagsProvider
bean, as shown in the following example:
@Configuration(proxyBeanMethods = false)
public class MyConnectionPoolTagsProviderConfiguration {
@Bean
public MongoConnectionPoolTagsProvider customConnectionPoolTagsProvider() {
return new CustomConnectionPoolTagsProvider();
}
}
To disable the auto-configured connection pool metrics, set the following property:
management.metrics.mongo.connectionpool.enabled=false
management:
metrics:
mongo:
connectionpool:
enabled: false
6.3.17. @Timed Annotation Support
The @Timed
annotation from the io.micrometer.core.annotation
package can be used with several of the supported technologies listed above.
If supported, the annotation can be used either at the class-level or the method-level.
For example, the following code shows how the annotation can be used to instrument all request mappings in a @RestController
:
@RestController
@Timed
public class MyController {
@GetMapping("/api/addresses")
public List<Address> listAddress() {
return ...
}
@GetMapping("/api/people")
public List<Person> listPeople() {
return ...
}
}
If you only want to instrument a single mapping, you can use the annotation on the method instead of the class:
@RestController
public class MyController {
@GetMapping("/api/addresses")
public List<Address> listAddress() {
return ...
}
@GetMapping("/api/people")
@Timed
public List<Person> listPeople() {
return ...
}
}
You can also combine class-level and method-level annotations if you want to change timing details for a specific method:
@RestController
@Timed
public class MyController {
@GetMapping("/api/addresses")
public List<Address> listAddress() {
return ...
}
@GetMapping("/api/people")
@Timed(extraTags = { "region", "us-east-1" })
@Timed(value = "all.people", longTask = true)
public List<Person> listPeople() {
return ...
}
}
A @Timed annotation with longTask = true will enable a long task timer for the method.
Long task timers require a separate metric name, and can be stacked with a short task timer.
|
6.4. Registering Custom Metrics
To register custom metrics, inject MeterRegistry
into your component, as shown in the following example:
@Component
public class MyBean {
private final Dictionary dictionary;
public MyBean(MeterRegistry registry) {
this.dictionary = Dictionary.load();
registry.gauge("dictionary.size", Tags.empty(), this.dictionary.getWords().size());
}
}
If your metrics depend on other beans, it is recommended that you use a MeterBinder
to register them, as shown in the following example:
public class MyMeterBinderConfiguration {
@Bean
public MeterBinder queueSize(Queue queue) {
return (registry) -> Gauge.builder("queueSize", queue::size).register(registry);
}
}
Using a MeterBinder
ensures that the correct dependency relationships are set up and that the bean is available when the metric’s value is retrieved.
A MeterBinder
implementation can also be useful if you find that you repeatedly instrument a suite of metrics across components or applications.
By default, metrics from all MeterBinder beans will be automatically bound to the Spring-managed MeterRegistry .
|
6.5. Customizing Individual Metrics
If you need to apply customizations to specific Meter
instances you can use the io.micrometer.core.instrument.config.MeterFilter
interface.
For example, if you want to rename the mytag.region
tag to mytag.area
for all meter IDs beginning with com.example
, you can do the following:
@Configuration(proxyBeanMethods = false)
public class MyMetricsFilterConfiguration {
@Bean
public MeterFilter renameRegionTagMeterFilter() {
return MeterFilter.renameTag("com.example", "mytag.region", "mytag.area");
}
}
By default, all MeterFilter beans will be automatically bound to the Spring-managed MeterRegistry .
Make sure to register your metrics using the Spring-managed MeterRegistry and not any of the static methods on Metrics .
These use the global registry that is not Spring-managed.
|
6.5.1. Common Tags
Common tags are generally used for dimensional drill-down on the operating environment like host, instance, region, stack, etc. Commons tags are applied to all meters and can be configured as shown in the following example:
management.metrics.tags.region=us-east-1
management.metrics.tags.stack=prod
management:
metrics:
tags:
region: "us-east-1"
stack: "prod"
The example above adds region
and stack
tags to all meters with a value of us-east-1
and prod
respectively.
The order of common tags is important if you are using Graphite.
As the order of common tags cannot be guaranteed using this approach, Graphite users are advised to define a custom MeterFilter instead.
|
6.5.2. Per-meter Properties
In addition to MeterFilter
beans, it’s also possible to apply a limited set of customization on a per-meter basis using properties.
Per-meter customizations apply to any all meter IDs that start with the given name.
For example, the following will disable any meters that have an ID starting with example.remote
management.metrics.enable.example.remote=false
management:
metrics:
enable:
example:
remote: false
The following properties allow per-meter customization:
Property | Description |
---|---|
|
Whether to deny meters from emitting any metrics. |
|
Whether to publish a histogram suitable for computing aggregable (across dimension) percentile approximations. |
|
Publish less histogram buckets by clamping the range of expected values. |
|
Publish percentile values computed in your application |
|
Publish a cumulative histogram with buckets defined by your service-level objectives. |
For more details on concepts behind percentiles-histogram
, percentiles
and slo
refer to the "Histograms and percentiles" section of the micrometer documentation.
6.6. Metrics Endpoint
Spring Boot provides a metrics
endpoint that can be used diagnostically to examine the metrics collected by an application.
The endpoint is not available by default and must be exposed, see exposing endpoints for more details.
Navigating to /actuator/metrics
displays a list of available meter names.
You can drill down to view information about a particular meter by providing its name as a selector, e.g. /actuator/metrics/jvm.memory.max
.
The name you use here should match the name used in the code, not the name after it has been naming-convention normalized for a monitoring system it is shipped to.
In other words, if |
You can also add any number of tag=KEY:VALUE
query parameters to the end of the URL to dimensionally drill down on a meter, e.g. /actuator/metrics/jvm.memory.max?tag=area:nonheap
.
The reported measurements are the sum of the statistics of all meters matching the meter name and any tags that have been applied.
So in the example above, the returned "Value" statistic is the sum of the maximum memory footprints of "Code Cache", "Compressed Class Space", and "Metaspace" areas of the heap.
If you only wanted to see the maximum size for the "Metaspace", you could add an additional |
7. Auditing
Once Spring Security is in play, Spring Boot Actuator has a flexible audit framework that publishes events (by default, “authentication success”, “failure” and “access denied” exceptions). This feature can be very useful for reporting and for implementing a lock-out policy based on authentication failures.
Auditing can be enabled by providing a bean of type AuditEventRepository
in your application’s configuration.
For convenience, Spring Boot offers an InMemoryAuditEventRepository
.
InMemoryAuditEventRepository
has limited capabilities and we recommend using it only for development environments.
For production environments, consider creating your own alternative AuditEventRepository
implementation.
7.1. Custom Auditing
To customize published security events, you can provide your own implementations of AbstractAuthenticationAuditListener
and AbstractAuthorizationAuditListener
.
You can also use the audit services for your own business events.
To do so, either inject the AuditEventRepository
bean into your own components and use that directly or publish an AuditApplicationEvent
with the Spring ApplicationEventPublisher
(by implementing ApplicationEventPublisherAware
).
8. HTTP Tracing
HTTP Tracing can be enabled by providing a bean of type HttpTraceRepository
in your application’s configuration.
For convenience, Spring Boot offers an InMemoryHttpTraceRepository
that stores traces for the last 100 request-response exchanges, by default.
InMemoryHttpTraceRepository
is limited compared to other tracing solutions and we recommend using it only for development environments.
For production environments, use of a production-ready tracing or observability solution, such as Zipkin or Spring Cloud Sleuth, is recommended.
Alternatively, create your own HttpTraceRepository
that meets your needs.
The httptrace
endpoint can be used to obtain information about the request-response exchanges that are stored in the HttpTraceRepository
.
9. Process Monitoring
In the spring-boot
module, you can find two classes to create files that are often useful for process monitoring:
-
ApplicationPidFileWriter
creates a file containing the application PID (by default, in the application directory with a file name ofapplication.pid
). -
WebServerPortFileWriter
creates a file (or files) containing the ports of the running web server (by default, in the application directory with a file name ofapplication.port
).
By default, these writers are not activated, but you can enable:
9.1. Extending Configuration
In the META-INF/spring.factories
file, you can activate the listener(s) that writes a PID file, as shown in the following example:
org.springframework.context.ApplicationListener=\ org.springframework.boot.context.ApplicationPidFileWriter,\ org.springframework.boot.web.context.WebServerPortFileWriter
10. Cloud Foundry Support
Spring Boot’s actuator module includes additional support that is activated when you deploy to a compatible Cloud Foundry instance.
The /cloudfoundryapplication
path provides an alternative secured route to all @Endpoint
beans.
The extended support lets Cloud Foundry management UIs (such as the web application that you can use to view deployed applications) be augmented with Spring Boot actuator information. For example, an application status page may include full health information instead of the typical “running” or “stopped” status.
The /cloudfoundryapplication path is not directly accessible to regular users.
In order to use the endpoint, a valid UAA token must be passed with the request.
|
10.1. Disabling Extended Cloud Foundry Actuator Support
If you want to fully disable the /cloudfoundryapplication
endpoints, you can add the following setting to your application.properties
file:
management.cloudfoundry.enabled=false
management:
cloudfoundry:
enabled: false
10.2. Cloud Foundry Self-signed Certificates
By default, the security verification for /cloudfoundryapplication
endpoints makes SSL calls to various Cloud Foundry services.
If your Cloud Foundry UAA or Cloud Controller services use self-signed certificates, you need to set the following property:
management.cloudfoundry.skip-ssl-validation=true
management:
cloudfoundry:
skip-ssl-validation: true
10.3. Custom Context Path
If the server’s context-path has been configured to anything other than /
, the Cloud Foundry endpoints will not be available at the root of the application.
For example, if server.servlet.context-path=/app
, Cloud Foundry endpoints will be available at /app/cloudfoundryapplication/*
.
If you expect the Cloud Foundry endpoints to always be available at /cloudfoundryapplication/*
, regardless of the server’s context-path, you will need to explicitly configure that in your application.
The configuration will differ depending on the web server in use.
For Tomcat, the following configuration can be added:
@Configuration(proxyBeanMethods = false)
public class MyCloudFoundryConfiguration {
@Bean
public TomcatServletWebServerFactory servletWebServerFactory() {
return new TomcatServletWebServerFactory() {
@Override
protected void prepareContext(Host host, ServletContextInitializer[] initializers) {
super.prepareContext(host, initializers);
StandardContext child = new StandardContext();
child.addLifecycleListener(new Tomcat.FixContextListener());
child.setPath("/cloudfoundryapplication");
ServletContainerInitializer initializer = getServletContextInitializer(getContextPath());
child.addServletContainerInitializer(initializer, Collections.emptySet());
child.setCrossContext(true);
host.addChild(child);
}
};
}
private ServletContainerInitializer getServletContextInitializer(String contextPath) {
return (classes, context) -> {
Servlet servlet = new GenericServlet() {
@Override
public void service(ServletRequest req, ServletResponse res) throws ServletException, IOException {
ServletContext context = req.getServletContext().getContext(contextPath);
context.getRequestDispatcher("/cloudfoundryapplication").forward(req, res);
}
};
context.addServlet("cloudfoundry", servlet).addMapping("/*");
};
}
}
11. What to Read Next
You might want to read about graphing tools such as Graphite.
Otherwise, you can continue on, to read about ‘deployment options’ or jump ahead for some in-depth information about Spring Boot’s build tool plugins.