This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.1.0!

Guide: Dynamic Tool Discovery

As an agent connects to more services, such as Slack, GitHub, Jira or several MCP servers, its tool library grows. Every tool definition is sent to the model with every request, which costs tokens, and models choose less reliably among many similar tools.

The tool search pattern, described by Anthropic, solves this: the model first receives only a search tool, searches for the capabilities it needs, and then receives only the definitions of the tools it found. Spring AI implements it with the ToolSearchToolCallingAdvisor, which works with every chat model that supports tool calling.

This guide shows the setup step by step. Tool Search is the reference for all options.

How It Works

ToolSearchToolCallingAdvisor extends the ToolCallingAdvisor, so it runs the tool calling loop too:

Tool search flow
  1. Indexing. The tools of the request are indexed in a ToolIndex, once per session. They are indexed again only when the set of tools changes.

  2. First request. The model receives only the definition of the search tool.

  3. Search. When the model needs a capability, it calls the search tool with a description of what it needs.

  4. Expansion. The ToolIndex returns the matching tools, and the advisor adds their definitions to the tools of the next request.

  5. Tool calls. The model calls the tools it found, and the loop continues until the model answers.

Setup with Spring Boot

Add the starter:

<dependency>
	<groupId>org.springframework.ai</groupId>
	<artifactId>spring-ai-starter-tool-search-advisor</artifactId>
</dependency>

Enable the advisor:

spring.ai.chat.client.tool-search-advisor.enabled=true

The auto-configured ChatClient.Builder then uses ToolSearchToolCallingAdvisor in place of the default tool calling advisor, with a regular-expression index by default. Register your tools as usual:

@Bean
ChatClient chatClient(ChatClient.Builder builder) {
	return builder
		.defaultTools(new MyTools()) // indexed, but not sent to the model up front
		.build();
}

Setup without Spring Boot

Add spring-ai-tool-search-advisor, choose a ToolIndex, and register the advisor yourself:

var advisor = ToolSearchToolCallingAdvisor.builder()
	.toolIndex(new LuceneToolIndex())
	.build();

ChatClient chatClient = ChatClient.builder(chatModel)
	.defaultTools(new MyTools())
	.defaultAdvisors(advisor) // replaces the default ToolCallingAdvisor
	.build();

LuceneToolIndex needs lucene-core on the classpath, and VectorToolIndex needs spring-ai-vector-store: the module declares them as optional dependencies. RegexToolIndex needs nothing more.

Calling the Model

The advisor keeps an index for each session, so every call needs a session id. It reads it from the same parameter as the memory advisors:

String answer = chatClient.prompt()
	.user("""
		Help me plan what to wear today in Amsterdam.
		Please suggest clothing shops that are open right now.
		""")
	.advisors(a -> a.param(MemoryAdvisor.CONVERSATION_ID, "session-42"))
	.call()
	.content();

When you use session memory, SessionAdvisors.session(sessionId) sets the same parameter. A call without a session id fails with an IllegalArgumentException.

The tools in this example:

class MyTools {

	@Tool(description = "Get the weather for a given location at a given time")
	String weather(String location, @ToolParam(description = "YYYY-MM-DDTHH:mm") String atTime) {
		return weatherService.forecast(location, atTime);
	}

	@Tool(description = "Get the names of clothing shops in a location that are open at a given time")
	List<String> clothingShops(String location, @ToolParam(description = "YYYY-MM-DDTHH:mm") String openAt) {
		return shopDirectory.openShops(location, openAt);
	}

	@Tool(description = "Get the current date and time in a location")
	String currentTime(String location) {
		return clock.now(location);
	}
}

Choosing an Index

Index Implementation Finds tools by

Regular expression (default with the starter)

RegexToolIndex

The words of the search query, matched in tool names and descriptions. No extra dependencies.

Keyword

LuceneToolIndex

Full-text search over tool names and descriptions.

Semantic

VectorToolIndex

Similarity of the query to the tool descriptions, using a VectorStore.

Decision

DecisionToolIndex

A decision model that judges which tools apply to the query.

See Tool Search for the details of each index.

Tool search pays off when the tool definitions are large compared with the rest of the request, for example with tens of tools or several MCP servers, or when the model picks the wrong tool among many similar ones. With a few compact tools that are used in every session, the extra search round costs more than it saves.

The announcement blog post reports benchmarks with 28 tools, in which tool search reduced the tokens per request by 34% with OpenAI, 60% with Gemini and 64% with Anthropic. Your results depend on your tools and models.