This version is still in development and is not considered stable yet. For the latest snapshot version, please use Spring AI 1.0.3!

Recursive Advisors

What is a Recursive Advisor?

Advisors Recursive Recursive advisors are a special type of advisor that can loop through the downstream advisor chain multiple times. This pattern is useful when you need to repeatedly call the LLM until a certain condition is met, such as:

Executing tool calls in a loop until no more tools need to be called
Validating structured output and retrying if validation fails
Implementing Evaluation logic with modifications to the request
Implementing retry logic with modifications to the request

The CallAdvisorChain.copy(CallAdvisor after) method is the key utility that enables recursive advisor patterns. It creates a new advisor chain that contains only the advisors that come after the specified advisor in the original chain and allows the recursive advisor to call this sub-chain as needed. This approach ensures that:

The recursive advisor can loop through the remaining advisors in the chain
Other advisors in the chain can observe and intercept each iteration
The advisor chain maintains proper ordering and observability
The recursive advisor doesn’t re-execute advisors that came before it

Built-in Recursive Advisors

Spring AI provides two built-in recursive advisors that demonstrate this pattern:

ToolCallAdvisor

The ToolCallAdvisor implements the tool calling loop as part of the advisor chain, rather than relying on the model’s internal tool execution. This enables other advisors in the chain to intercept and observe the tool calling process.

Key features:

Disables the model’s internal tool execution by setting setInternalToolExecutionEnabled(false)
Loops through the advisor chain until no more tool calls are present
Supports "return direct" functionality - when a tool execution has returnDirect=true, it interrupts the tool calling loop and returns the tool execution result directly to the client application instead of sending it back to the LLM
Uses callAdvisorChain.copy(this) to create a sub-chain for recursive calls
Includes null safety checks to handle cases where the chat response might be null

Example usage:

var toolCallAdvisor = ToolCallAdvisor.builder()
    .toolCallingManager(toolCallingManager)
    .advisorOrder(BaseAdvisor.HIGHEST_PRECEDENCE + 300)
    .build();

var chatClient = ChatClient.builder(chatModel)
    .defaultAdvisors(toolCallAdvisor)
    .build();

Return Direct Functionality

The "return direct" feature allows tools to bypass the LLM and return their results directly to the client application. This is useful when:

The tool’s output is the final answer and doesn’t need LLM processing
You want to reduce latency by avoiding an additional LLM call
The tool result should be returned as-is without interpretation

When a tool execution has returnDirect=true, the ToolCallAdvisor will:

Execute the tool call as normal
Detect the returnDirect flag in the ToolExecutionResult
Break out of the tool calling loop
Return the tool execution result directly to the client application as a ChatResponse with the tool’s output as the generation content

StructuredOutputValidationAdvisor

The StructuredOutputValidationAdvisor validates the structured JSON output against a generated JSON schema and retries the call if validation fails, up to a specified number of attempts.

Key features:

Automatically generates a JSON schema from the expected output type
Validates the LLM response against the schema
Retries the call if validation fails, up to a configurable number of attempts
Augments the prompt with validation error messages on retry attempts to help the LLM correct its output
Uses callAdvisorChain.copy(this) to create a sub-chain for recursive calls
Optionally supports a custom ObjectMapper for JSON processing

Example usage:

var validationAdvisor = StructuredOutputValidationAdvisor.builder()
    .outputType(MyResponseType.class)
    .maxRepeatAttempts(3)
    .advisorOrder(BaseAdvisor.HIGHEST_PRECEDENCE + 1000)
    .build();

var chatClient = ChatClient.builder(chatModel)
    .defaultAdvisors(validationAdvisor)
    .build();