|
This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.1.0! |
Advisors
Every ChatClient call is executed by a chain of advisors.
An advisor is a component that runs around the call to the model: it can change the request before the model sees it, change the response before your code sees it, call the model more than once, or answer without calling it at all.
Advisors are to ChatClient what filters are to a servlet container.
Most features of ChatClient are advisors:
session memory, retrieval augmented generation, the tool calling loop, schema validation, guardrails and logging.
Even the call to the model is made by an advisor, the last one in the chain.
To understand what a ChatClient call does, look at its advisor chain.
For an agent built on a ChatClient, this chain is the backbone of its harness, the part of an agent that isn’t the model.
The Advisor Chain
For each call, ChatClient builds a chain:
-
It takes the advisors registered on the builder with
defaultAdvisors(…)and those added to the call withadvisors(…). -
It adds a
ToolCallingAdvisor, unless the chain already has one, or auto-registration is disabled. It does this even when the call has no tools, because another advisor can add tools while the call runs. -
It adds
ChatModelCallAdvisorandChatModelStreamAdvisorat the end. They add the format instructions of.entity(…)to the prompt, or the schema for native structured output, and call the chat model. -
It sorts the advisors by their order.
The following chain uses session memory, retrieval and schema validation:
The request travels down the chain as a ChatClientRequest, and the response travels back up as a ChatClientResponse.
Each advisor decides what to do:
-
Change the request or the response.
QuestionAnswerAdvisoradds documents to the prompt, andSessionTurnAdvisorstores the answer. -
Run the rest of the chain more than once.
ToolCallingAdvisorruns everything after it again for each tool round, andStructuredOutputValidationAdvisordoes so when an answer does not match its schema. See Recursive Advisors. -
Answer without calling the rest of the chain.
SafeGuardAdvisorreturns a refusal when the input contains a blocked word, andSemanticCacheAdvisorreturns a cached answer.
Order
An advisor’s order, from getOrder(), decides its place in the chain:
-
The advisor with the lowest order runs first on the way to the model, and last on the way back. The chain works like a stack.
-
Ordered.HIGHEST_PRECEDENCEis the lowest value andOrdered.LOWEST_PRECEDENCEthe highest. -
Advisors with the same order run in an undefined order.
The position relative to the tool loop matters most.
An advisor before ToolCallingAdvisor runs once per call.
An advisor after it runs again for every tool round, and sees each intermediate request and response.
For example, SessionTurnAdvisor is placed before the loop so that it records each turn once, and SessionMemoryAdvisor after it so that every round sees the conversation, including the tool calls of earlier rounds.
| Advisor | Default order | Position |
|---|---|---|
|
|
Before the tool loop |
|
|
The tool loop |
|
|
Inside the tool loop |
|
|
Inside the tool loop |
|
|
Close to the model, so that a retry repeats only the model call |
|
|
Last: call the model |
The built-in advisors let you change their order through their builders.
When ChatClient auto-registers the ToolCallingAdvisor and finds a MemoryAdvisor after it in the chain, such as SessionMemoryAdvisor, it turns off the advisor’s internal conversation history and leaves the history of the tool rounds to that memory advisor.
If you register the ToolCallingAdvisor yourself, call disableInternalConversationHistory() on its builder to get the same behavior.
Registering Advisors
Register the advisors that every call needs once, on the builder. Add the ones that a single call needs to that call:
ChatClient chatClient = ChatClient.builder(chatModel)
.defaultAdvisors(SessionAdvisors.builder(sessionService).build()) // the session memory advisors
.defaultAdvisors(QuestionAnswerAdvisor.builder(vectorStore).build()) // retrieval
.build();
String answer = chatClient.prompt()
.user(userText)
.advisors(SessionAdvisors.session(sessionId)) // a parameter for the session advisors
.advisors(new SimpleLoggerAdvisor()) // an advisor for this call only
.call()
.content();
advisors(…) also passes parameters to the advisors of the chain.
SessionAdvisors.session(sessionId) is a shortcut for advisors(a → a.param(SessionMemoryAdvisor.SESSION_ID_CONTEXT_KEY, sessionId)), whose key is the same as MemoryAdvisor.CONVERSATION_ID.
Each advisor documents the parameters it reads.
Advisors and Memory in ChatClient shows more configuration, such as logging and the options of the tool calling advisor.
The Request, the Response and the Context
A ChatClientRequest holds the Prompt and a context map, and a ChatClientResponse holds the ChatResponse and the same context.
ChatClient creates the context for each call from the parameters given with advisors(…), and adds its own entries, such as the output format of .entity(…).
Advisors use the context to pass information to each other and to themselves: an advisor can put an entry into the context on the way down and read it on the way back up.
The mutate() method of the request and of the response creates a changed copy, for example with an augmented prompt.
The Tool Calling Advisor
ToolCallingAdvisor is the only advisor that ChatClient adds on its own, apart from the advisors that call the model.
A chain can have at most one advisor that implements ToolAdvisor, so adding your own ToolCallingAdvisor, or a subclass such as ToolSearchToolCallingAdvisor, replaces the default one.
To disable the automatic registration, for example to run the tool loop yourself:
-
for every call of the auto-configured
ChatClient, setspring.ai.chat.client.tool-calling.enabled=false; -
for one call, add
.advisors(AdvisorParams.toolCallingAdvisorAutoRegister(false)).
See ToolCallingAdvisor and The Tool Calling Loop.
Observability
Each advisor in the chain is observed: a spring.ai.advisor observation measures the time spent in the advisor, including the advisors after it, and carries its name and order.
In a trace, the advisor chain of a call appears as nested spans.
See Observability.
The Advisor API
The advisor interfaces are in the org.springframework.ai.chat.client.advisor.api package:
-
CallAdvisoradvisescall(), andStreamAdvisoradvisesstream(). An advisor that should work for both implements both. -
BaseAdvisorimplements both from two methods,before(…)andafter(…), for advisors that change the request and the response but do not need to control the call of the rest of the chain. -
CallAdvisorChainandStreamAdvisorChaincall the next advisor. Theircopy(after)method creates a chain of the advisors that follow a given advisor, which recursive advisors call more than once. -
ToolAdvisorandMemoryAdvisorare marker interfaces.ChatClientuses them to find the tool calling advisor and the memory advisors in a chain.
Writing an Advisor shows how to implement them.