|
This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.1.0! |
Metrics Reference
This page lists the metrics that Spring AI’s observations produce, with the names they have in Prometheus. Observability describes the observations and their keys.
How Observations Become Metrics
Spring Boot records two timers for every observation:
-
A timer with the name of the observation, for the operations that have completed. It has a tag for each low-cardinality key of the observation, and an
errortag with the exception class name, ornonewhen the operation succeeded. -
A long task timer with the name of the observation followed by
.active, for the operations that are still running.
Prometheus replaces the dots of a name with underscores and adds the unit and a suffix.
For the gen_ai.client.operation observation, the series are:
| Series | Meaning |
|---|---|
|
Number of completed operations |
|
Total time of the completed operations |
|
Longest completed operation in the recent time window |
|
Number of operations running now |
|
Time the running operations have taken so far |
|
Time the longest running operation has taken so far |
The other observations follow the same pattern. Other registries name the series their own way; with OTLP, for example, the timer keeps the name of the observation.
Timers
| Series prefix | Records | Tags (besides error) |
|---|---|---|
|
|
|
|
Each advisor of a |
|
|
Calls of chat, embedding, image, text-to-speech, transcription and decision models |
|
|
Tool calls |
|
|
Vector store operations |
|
The timers nest like the spans of a trace.
The spring_ai_chat_client timer of a call includes the time of its advisors, and each spring_ai_advisor timer includes the time of the advisors it calls, so also of the model calls and tool calls inside it.
Use gen_ai_client_operation to measure the time spent in the model.
Token Usage
Model calls increment the gen_ai.client.token.usage counter by the number of tokens the provider reports, exported as gen_ai_client_token_usage_total.
| Tag | Meaning |
|---|---|
|
|
|
The low-cardinality keys of the model call, as on the |
A provider that reports no token usage for a call adds nothing to the counter.
Sum either input and output or total, not all three.
Example Queries
Tokens per minute, per model and token type:
sum by (gen_ai_request_model, gen_ai_token_type) (
rate(gen_ai_client_token_usage_total{gen_ai_token_type!="total"}[5m])
) * 60
Average duration of chat model calls, per provider:
sum by (gen_ai_system) (rate(gen_ai_client_operation_seconds_sum{gen_ai_operation_name="chat"}[5m]))
/
sum by (gen_ai_system) (rate(gen_ai_client_operation_seconds_count{gen_ai_operation_name="chat"}[5m]))
Share of failed tool calls, per tool:
sum by (spring_ai_tool_definition_name) (rate(spring_ai_tool_seconds_count{error!="none"}[5m]))
/
sum by (spring_ai_tool_definition_name) (rate(spring_ai_tool_seconds_count[5m]))
Model calls in progress:
sum by (gen_ai_system) (gen_ai_client_operation_active_seconds_count)
Percentiles need histogram buckets, which Spring Boot does not publish by default. Turn them on for a timer with a property:
management.metrics.distribution.percentiles-histogram.gen_ai.client.operation=true
Then the 95th percentile of the model call duration is:
histogram_quantile(0.95,
sum by (le, gen_ai_system) (rate(gen_ai_client_operation_seconds_bucket[5m])))