|
This version is still in development and is not considered stable yet. For the latest stable version, please use Spring Boot 4.0.6! |
Observability
Observability is the ability to observe the internal state of a running system from the outside. It consists of the three pillars: logging, metrics and traces.
For metrics and traces, Spring Boot uses Micrometer Observation.
To create your own observations (which will lead to metrics and traces), you can inject an ObservationRegistry.
-
Java
-
Kotlin
import io.micrometer.observation.Observation;
import io.micrometer.observation.ObservationRegistry;
import org.springframework.stereotype.Component;
@Component
public class MyCustomObservation {
private final ObservationRegistry observationRegistry;
public MyCustomObservation(ObservationRegistry observationRegistry) {
this.observationRegistry = observationRegistry;
}
public void doSomething() {
Observation.createNotStarted("doSomething", this.observationRegistry)
.lowCardinalityKeyValue("locale", "en-US")
.highCardinalityKeyValue("userId", "42")
.observe(() -> {
// Execute business logic here
});
}
}import io.micrometer.observation.Observation
import io.micrometer.observation.ObservationRegistry;
import org.springframework.stereotype.Component
@Component
class MyCustomObservation(private val observationRegistry: ObservationRegistry) {
fun doSomething() {
Observation.createNotStarted("doSomething", observationRegistry)
.lowCardinalityKeyValue("locale", "en-US")
.highCardinalityKeyValue("userId", "42")
.observe {
// Execute business logic here
}
}
}| Low cardinality tags will be added to metrics and traces, while high cardinality tags will only be added to traces. |
Beans of type ObservationPredicate, GlobalObservationConvention, ObservationFilter and ObservationHandler will be automatically registered on the ObservationRegistry.
You can additionally register any number of ObservationRegistryCustomizer beans to further configure the registry.
| Observability for JDBC can be configured using a separate project. The Datasource Micrometer project provides a Spring Boot starter which automatically creates observations when JDBC operations are invoked. Read more about it in the reference documentation. |
Observability for R2DBC is built into Spring Boot.
To enable it, add the io.r2dbc:r2dbc-proxy dependency to your project.
|
Context Propagation
Observability support relies on the Context Propagation library for forwarding the current observation across threads and reactive pipelines.
By default, ThreadLocal values are not automatically reinstated in reactive operators.
This behavior is controlled with the spring.reactor.context-propagation property, which can be set to auto to enable automatic propagation.
If you’re working with @Async methods and the AsyncTaskExecutor is auto-configured, you have to opt-in for context propagation using the spring.task.execution.propagate-context property.
If you are configuring the AsyncTaskExecutor yourself, then you need to register a ContextPropagatingTaskDecorator bean, as shown in the following example:
-
Java
-
Kotlin
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.support.ContextPropagatingTaskDecorator;
@Configuration(proxyBeanMethods = false)
class ContextPropagationConfiguration {
@Bean
ContextPropagatingTaskDecorator contextPropagatingTaskDecorator() {
return new ContextPropagatingTaskDecorator();
}
}import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.core.task.support.ContextPropagatingTaskDecorator
@Configuration(proxyBeanMethods = false)
class ContextPropagationConfiguration {
@Bean
fun contextPropagatingTaskDecorator(): ContextPropagatingTaskDecorator {
return ContextPropagatingTaskDecorator()
}
}For more details about observations please see the Micrometer Observation documentation.
Common Tags
Common tags are generally used for dimensional drill-down on the operating environment, such as host, instance, region, stack, and others. Common tags are applied to all observations as low cardinality tags and can be configured, as the following example shows:
-
Properties
-
YAML
management.observations.key-values.region=us-east-1
management.observations.key-values.stack=prodmanagement:
observations:
key-values:
region: "us-east-1"
stack: "prod"The preceding example adds region and stack tags to all observations with a value of us-east-1 and prod, respectively.
Preventing Observations
If you’d like to prevent some observations from being reported, you can use the management.observations.enable properties:
-
Properties
-
YAML
management.observations.enable.denied.prefix=false
management.observations.enable.another.denied.prefix=falsemanagement:
observations:
enable:
denied:
prefix: false
another:
denied:
prefix: falseThe preceding example will prevent all observations with a name starting with denied.prefix or another.denied.prefix.
If you want to prevent Spring Security from reporting observations, set the property management.observations.enable.spring.security to false.
|
If you need greater control over the prevention of observations, you can register beans of type ObservationPredicate.
Observations are only reported if all the ObservationPredicate beans return true for that observation.
-
Java
-
Kotlin
import io.micrometer.observation.Observation.Context;
import io.micrometer.observation.ObservationPredicate;
import org.springframework.stereotype.Component;
@Component
class MyObservationPredicate implements ObservationPredicate {
@Override
public boolean test(String name, Context context) {
return !name.contains("denied");
}
}import io.micrometer.observation.Observation.Context
import io.micrometer.observation.ObservationPredicate
import org.springframework.stereotype.Component
@Component
class MyObservationPredicate : ObservationPredicate {
override fun test(name: String, context: Context): Boolean {
return !name.contains("denied")
}
}The preceding example will prevent all observations whose name contains "denied".
Micrometer Observation Annotations support
To enable scanning of observability annotations like @Observed, @Timed, @Counted, @MeterTag and @NewSpan, set the management.observations.annotations.enabled property to true.
A dependency on org.aspectj:aspectjweaver, which is part of spring-boot-starter-aspectj, is also required.
This feature is supported by Micrometer directly.
Please refer to the Micrometer, Micrometer Observation and Micrometer Tracing reference docs.
When you annotate methods or classes which are already instrumented (for example, Spring Data repositories or Spring MVC controllers), you will get duplicate observations.
In that case you can either disable the automatic instrumentation using properties or an ObservationPredicate and rely on your annotations, or you can remove your annotations.
|
OpenTelemetry Support
| There are several ways to support OpenTelemetry in your application. You can use the OpenTelemetry Java Agent or the OpenTelemetry Spring Boot Starter, which are supported by the OTel community; the metrics and traces use the semantic conventions defined by OTel libraries. This documentation describes OpenTelemetry as officially supported by the Spring team, using Micrometer and the OTLP exporter; the metrics and traces use the semantic conventions described in the Spring projects documentation, such as Spring Framework. |
Spring Boot’s actuator module includes basic support for OpenTelemetry.
It provides a bean of type OpenTelemetry, and if there are beans of type SdkTracerProvider, ContextPropagators, SdkLoggerProvider or SdkMeterProvider in the application context, they automatically get registered.
Additionally, it provides a Resource bean.
The attributes of the auto-configured Resource can be configured via the management.opentelemetry.resource-attributes configuration property.
Auto-configured attributes will be merged with attributes from the OTEL_RESOURCE_ATTRIBUTES and OTEL_SERVICE_NAME environment variables, with attributes configured through the configuration property taking precedence over those from the environment variables.
If you have defined your own Resource bean, this will no longer be the case.
| Spring Boot does not provide automatic exporting of OpenTelemetry metrics or logs. Exporting OpenTelemetry traces is only auto-configured when used together with Micrometer Tracing. |
Disabling OpenTelemetry
The OpenTelemetry support can be disabled by setting the management.opentelemetry.enabled property to false.
This behaves similarly to the OTEL_SDK_DISABLED environment variable (but negated): when the SDK is disabled, metrics, traces, and logging will use no-op implementations.
Context propagators are not affected and continue to function normally.
| Keep in mind that Spring Boot doesn’t use OpenTelemetry’s metrics functionality, so metrics might still be enabled even when disabling OpenTelemetry. |
Environment variables
Spring Boot supports a subset of the OpenTelemetry SDK environment variables.
These environment variables are automatically mapped to Spring Boot configuration properties at startup.
When an environment variable has a signal-specific variant (for example, OTEL_EXPORTER_OTLP_TRACES_ENDPOINT) and a general variant (OTEL_EXPORTER_OTLP_ENDPOINT), the signal-specific variant takes precedence and is used as-is.
When the general variant is used as a fallback, the signal-specific path (v1/traces, v1/metrics, or v1/logs) is appended to it.
For example, setting OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4318 results in collector:4318/v1/traces for traces.
This mapping can be disabled by setting management.opentelemetry.map-environment-variables to false.
General
| Environment Variable | Spring Boot Property |
|---|---|
|
|
|
|
|
|
|
|
|
|
Metrics Exporter
| Environment Variable | Spring Boot Property |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Traces Exporter
| Environment Variable | Spring Boot Property |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Logs Exporter
| Environment Variable | Spring Boot Property |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Resource
The following environment variables configure the OpenTelemetry resource:
The OTEL_RESOURCE_ATTRIBUTES environment variable consists of a list of key-value pairs.
For example: key1=value1,key2=value2,key3=spring%20boot.
All attribute values are treated as strings, and any characters outside the baggage-octet range must be percent-encoded.
|
Unsupported Environment Variables
Other environment variables as described in the OpenTelemetry documentation are not supported.
If you want all environment variables specified by OpenTelemetry’s SDK to be effective, you have to supply your own OpenTelemetry bean.
| Doing this will switch off Spring Boot’s OpenTelemetry auto-configuration and may break the built-in observability functionality. |
First, add a dependency to io.opentelemetry:opentelemetry-sdk-extension-autoconfigure to get OpenTelemetry’s zero-code SDK autoconfigure module, then add this configuration:
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration(proxyBeanMethods = false)
class AutoConfiguredOpenTelemetrySdkConfiguration {
@Bean
OpenTelemetry autoConfiguredOpenTelemetrySdk() {
return AutoConfiguredOpenTelemetrySdk.initialize().getOpenTelemetrySdk();
}
}Logging
The OpenTelemetryLoggingAutoConfiguration configures OpenTelemetry’s SdkLoggerProvider.
Exporting logs via OTLP is supported through the OtlpLoggingAutoConfiguration, which enables OTLP log exporting over HTTP or gRPC.
If you need to apply advanced customizations to OTLP log record exporters, consider registering OtlpHttpLogRecordExporterBuilderCustomizer or OtlpGrpcLogRecordExporterBuilderCustomizer beans.
These will be invoked before the creation of the OtlpHttpLogRecordExporter or OtlpGrpcLogRecordExporter.
The customizers take precedence over anything applied by the auto-configuration.
|
However, while there is a SdkLoggerProvider bean, Spring Boot doesn’t support bridging logs to this bean out of the box.
This can be done with 3rd-party log bridges, as described in the Logging with OpenTelemetry section.
Metrics
The choice of metrics in the Spring portfolio is Micrometer, which means that metrics are not collected and exported through the OpenTelemetry’s SdkMeterProvider.
Spring Boot doesn’t provide a SdkMeterProvider bean.
However, Micrometer metrics can be exported via OTLP to any OpenTelemetry capable backend using the OtlpMeterRegistry, as described in the Metrics with OTLP section.
Micrometer’s OTLP registry doesn’t use the Resource bean, but setting OTEL_RESOURCE_ATTRIBUTES, OTEL_SERVICE_NAME or management.opentelemetry.resource-attributes works.
|
Metrics via the OpenTelemetry API and SDK
If you or a dependency you include make use of OpenTelemetry’s MeterProvider, those metrics are not exported.
We strongly recommend that you report your metrics with Micrometer.
If a dependency you include uses OpenTelemetry’s MeterProvider, you can include this configuration in your application to configure a MeterProvider bean, which you then have to wire into your dependency:
import java.time.Duration;
import io.opentelemetry.exporter.otlp.http.metrics.OtlpHttpMetricExporter;
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.export.MetricExporter;
import io.opentelemetry.sdk.metrics.export.MetricReader;
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;
import io.opentelemetry.sdk.resources.Resource;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration(proxyBeanMethods = false)
class OpenTelemetryMetricsConfiguration {
@Bean
OtlpHttpMetricExporter metricExporter() {
String endpoint = "http://localhost:4318/v1/metrics";
return OtlpHttpMetricExporter.builder().setEndpoint(endpoint).build();
}
@Bean
PeriodicMetricReader metricReader(MetricExporter exporter) {
Duration interval = Duration.ofMinutes(1);
return PeriodicMetricReader.builder(exporter).setInterval(interval).build();
}
@Bean
SdkMeterProvider meterProvider(Resource resource, MetricReader metricReader) {
return SdkMeterProvider.builder().registerMetricReader(metricReader).setResource(resource).build();
}
}This configuration also enables metrics export via OTLP over HTTP.
Tracing
If Micrometer tracing is used, the OpenTelemetryTracingAutoConfiguration configures OpenTelemetry’s SdkTracerProvider.
Exporting traces through OTLP is enabled by the OtlpTracingAutoConfiguration, which supports exporting traces with OTLP over HTTP or gRPC.
We strongly recommend using the Micrometer Observation or Tracing API instead of using the OpenTelemetry API directly.
