Tool Search Tool

As AI agents connect to more services — Slack, GitHub, Jira, MCP servers — tool libraries grow rapidly. A typical multi-server setup can easily aggregate 50+ tools consuming 55,000+ tokens before any conversation starts. Tool selection accuracy also degrades when models face 30+ similarly-named tools.

ToolSearchToolCallingAdvisor solves this by replacing the default ToolCallingAdvisor with an implementation of the progressive tool disclosure pattern: tool definitions are exposed to the model incrementally, on demand, rather than sent upfront. Benchmarks across OpenAI, Anthropic, and Gemini show 34–64% token reduction while maintaining access to large tool catalogs — see the Smart Tool Selection blog post for measurements and methodology.

This page is the reference for the advisor, its ToolIndex strategies, configuration, and Spring Boot auto-configuration. For the conceptual placement of this advisor in the broader tool calling architecture, see Scaling to Hundreds of Tools.

How It Works

Tool Search Tool Calling Flow

ToolSearchToolCallingAdvisor extends ToolCallingAdvisor and overrides the loop’s initialization and per-iteration hooks. The runtime flow:

  1. Indexing — at session start, all registered tools are indexed in the configured ToolIndex. No tool definitions are sent to the model.

  2. Initial request — the first request to the LLM contains only the built-in toolSearchTool definition.

  3. Discovery call — when the model needs a capability, it calls toolSearchTool with a natural-language query.

  4. Search & expand — the ToolIndex finds matching tools; their definitions are appended to the conversation for the next iteration.

  5. Tool invocation — the model, now equipped with the relevant definition, issues a normal tool call.

  6. Tool execution — ToolCallingManager executes the discovered tool and returns its result.

  7. Response — the model produces the final answer using the tool result.

The indexed tool set is scoped per session (see Session Scoping); concurrent conversations have isolated indexes.

When to Use

Good fit:

  • 10+ tools registered with the ChatClient.

  • Tool definitions consuming more than 10K tokens per request.

  • Multi-server MCP setups where the aggregated tool catalog is large.

  • Symptoms of tool-selection accuracy issues with large tool sets.

Stick with the default ToolCallingAdvisor when:

  • Your tool library is small (under 10 tools).

  • All tools are frequently used in every session.

  • Tool definitions are very compact (the search round-trips would outweigh the token savings).

Installation

Use the Spring Boot starter for the simplest setup (includes Lucene and auto-configuration):

  • Maven

  • Gradle

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-tool-search-advisor</artifactId>
</dependency>
dependencies {
    implementation 'org.springframework.ai:spring-ai-starter-tool-search-advisor'
}

Or use the library directly for manual configuration:

  • Maven

  • Gradle

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-tool-search-advisor</artifactId>
</dependency>
dependencies {
    implementation 'org.springframework.ai:spring-ai-tool-search-advisor'
}

Quick Start

The fastest path is the auto-configuration — see Spring Boot Auto-Configuration below. For manual wiring:

// 1. Configure a ToolIndex (semantic, keyword, or regex)
@Bean
ToolIndex toolIndex(VectorStore vectorStore) {
    return new VectorToolIndex(vectorStore);
}

// 2. Build the advisor
var toolSearchAdvisor = ToolSearchToolCallingAdvisor.builder()
    .toolIndex(toolIndex)
    .maxResults(5)
    .build();

// 3. Register with ChatClient — tools are indexed but NOT sent to the LLM up front
ChatClient chatClient = ChatClient.builder(chatModel)
    .defaultTools(new MyTools())
    .defaultAdvisors(toolSearchAdvisor)
    .build();

// 4. Make a request — supply a session ID via the advisor context
String answer = chatClient.prompt("Help me plan what to wear today in Amsterdam")
    .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, "user-42-session"))
    .call()
    .content();

Session Scoping

ToolSearchToolCallingAdvisor indexes tools per session. The session ID determines which tool index a request sees — this enables multi-tenant and multi-conversation isolation.

The caller must supply a session ID with every request. By default the advisor reads the session ID from the advisor context under the ChatMemory.CONVERSATION_ID key:

chatClient.prompt()
    .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, "user-42-session"))
    .user("...")
    .call()
    .content();

If your application already passes a session identifier under a different key — for example tenantId or userId — change the lookup key via sessionIdKeyName(…​) (or the corresponding property):

var advisor = ToolSearchToolCallingAdvisor.builder()
    .toolIndex(toolIndex)
    .sessionIdKeyName("tenantId")
    .build();
If memory advisors are configured with conversation IDs (the standard pattern with MessageChatMemoryAdvisor), the same key is already in the context — you get session scoping "for free" by virtue of the memory setup.

Search Strategies

The ToolIndex interface abstracts the search implementation. Three strategies are provided out of the box:

Strategy Implementation Best for

Semantic

VectorToolIndex

Natural-language queries, fuzzy matching, novel phrasings — when callers describe what they need rather than naming the tool

Keyword

LuceneToolIndex

Exact-term matching, fast retrieval, known vocabulary

Regex

RegexToolIndex

Tool name patterns (e.g. get_*_data); lightweight default with no dependencies

VectorToolIndex (Semantic)

Uses embedding-based similarity search. Best when callers describe what they need in natural language.

@Bean
ToolIndex vectorToolIndex(VectorStore vectorStore) {
    return new VectorToolIndex(vectorStore);
}

Requires a VectorStore bean (e.g. via spring-ai-starter-vector-store-pgvector). Tool name and description are embedded on indexing; queries from toolSearchTool are embedded and the top-K matches are returned.

LuceneToolIndex (Keyword)

Uses Apache Lucene for keyword-based search. Fast, no embedding model required.

@Bean
ToolIndex luceneToolIndex() {
    return new LuceneToolIndex();          // default minimum score 0.25
    // return new LuceneToolIndex(0.4f);   // custom minimum score threshold
}

Hits below the minimum score threshold are silently dropped. Raise the threshold to be more selective; lower it to be more permissive.

RegexToolIndex (Pattern)

Uses regex pattern matching against tool names. Useful when tool names follow a strict naming convention (e.g. get_*, database). Zero additional dependencies.

@Bean
ToolIndex regexToolIndex() {
    return new RegexToolIndex();
}

The default index when no explicit tool-index-type is configured.

Configuration

ToolSearchToolCallingAdvisor.Builder extends ToolCallingAdvisor.Builder and adds search-specific options. See ToolCallingAdvisor Builder Options for inherited settings.

Option Description Default

toolIndex(ToolIndex)

The search implementation to use.

Required

maxResults(Integer)

Maximum tool references returned per toolSearchTool call. When null, the LLM decides (the built-in tool description hints at 5).

null

systemMessageSuffix(String)

Custom prompt suffix appended to the system message to instruct the model on how to use toolSearchTool.

Built-in template (see DEFAULT_SYSTEM_PROMPT_SUFFIX.md)

referenceToolNameAccumulation(boolean)

When true, tool names discovered across all prior toolSearchTool calls are accumulated and injected. When false, only the results from the most recent turn are used (including all parallel toolSearchTool calls within that turn).

true

sessionIdKeyName(String)

Advisor context key used to look up the conversation/session ID.

ChatMemory.CONVERSATION_ID

evictionStrategy(ToolIndexEvictionStrategy)

Determines when session tool indexes are freed. See Index Eviction.

LruEvictionStrategy(1000)

ToolIndex API

The ToolIndex interface and its companion types (ToolSearchRequest, ToolSearchResponse, ToolReference) live in the spring-ai-tool-search-tool module under org.springframework.ai.tool.toolsearch. The built-in implementations (LuceneToolIndex, VectorToolIndex, RegexToolIndex) are also in this module.

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-tool-search-tool</artifactId>
</dependency>
public interface ToolIndex {

    void indexTool(String sessionId, ToolReference toolReference);

    /** Default implementation loops over indexTool. */
    void indexTools(String sessionId, List<ToolReference> toolReferences);

    ToolSearchResponse search(ToolSearchRequest request);

    void clearIndex(String sessionId);
}

Every operation is scoped by sessionId. Implement ToolIndex directly when you need a custom search strategy — for example, a database-backed catalog with role-based filtering, or a cached remote tool registry.

Index Eviction

Per-session tool indexes consume memory. The ToolIndexEvictionStrategy decides when to free them.

By default (LruEvictionStrategy(1000)), up to 1,000 active sessions are retained and the least-recently-used session is evicted once the cap is exceeded. Call advisor.evictSession(sessionId) to release a session eagerly (e.g. on logout).

Eviction is evaluated lazily on each request — no background thread is required.

Five built-in strategies are provided:

Strategy Behavior

LruEvictionStrategy(maxSessions) (default)

Evicts the least-recently-used session once the number of active sessions exceeds maxSessions.

NeverEvictStrategy.INSTANCE

Never evicts automatically; indexes persist until evictSession() is called explicitly.

AlwaysEvictStrategy.INSTANCE

Clears a session’s index before every request, forcing a full re-index each turn. Useful for testing or when tool sets change every request.

TtlEvictionStrategy(duration)

Evicts sessions whose last-access time exceeds the given TTL.

CompositeEvictionStrategy(strategies…​)

Delegates to multiple strategies; evicts a session if any delegate requests it.

 
// Default: LRU cap of 1000 sessions — no configuration needed
var advisor = ToolSearchToolCallingAdvisor.builder()
    .toolIndex(toolIndex)
    .build();

// Never evict — manage session lifetime yourself
var advisor = ToolSearchToolCallingAdvisor.builder()
    .toolIndex(toolIndex)
    .evictionStrategy(NeverEvictStrategy.INSTANCE)
    .build();

// Always evict — re-index every request (useful for testing)
var advisor = ToolSearchToolCallingAdvisor.builder()
    .toolIndex(toolIndex)
    .evictionStrategy(AlwaysEvictStrategy.INSTANCE)
    .build();

// LRU with custom cap
var advisor = ToolSearchToolCallingAdvisor.builder()
    .toolIndex(toolIndex)
    .evictionStrategy(new LruEvictionStrategy(200))
    .build();

// Evict sessions idle for more than 30 minutes
var advisor = ToolSearchToolCallingAdvisor.builder()
    .toolIndex(toolIndex)
    .evictionStrategy(new TtlEvictionStrategy(Duration.ofMinutes(30)))
    .build();

// Combine: TTL + LRU cap
var advisor = ToolSearchToolCallingAdvisor.builder()
    .toolIndex(toolIndex)
    .evictionStrategy(new CompositeEvictionStrategy(
        new TtlEvictionStrategy(Duration.ofMinutes(30)),
        new LruEvictionStrategy(200)))
    .build();

Spring Boot Auto-Configuration

The spring-ai-starter-tool-search-advisor starter provides zero-boilerplate setup. Enable it with a single property:

spring.ai.chat.client.tool-search-advisor.enabled=true

When enabled, the auto-configuration:

  • Registers a ToolSearchToolCallingAdvisor.Builder bean typed as ToolCallingAdvisor.Builder<?>. This transparently replaces the default ToolCallingAdvisor thanks to the @ConditionalOnMissingBean guard on the default builder — no code changes to your ChatClient are needed. See Custom ToolAdvisor: Auto-Configuration Integration for the underlying mechanism.

  • Auto-registers a ToolIndex bean unless your application declares one explicitly.

ToolIndex Auto-Selection

Set spring.ai.chat.client.tool-search-advisor.tool-index-type to select the implementation:

Value Implementation Requirements

regex (default)

RegexToolIndex

No additional dependencies

lucene

LuceneToolIndex

org.apache.lucene:lucene-core on the classpath (bundled in the starter)

vector

VectorToolIndex

A VectorStore bean in the application context

A custom ToolIndex bean declared by the application always takes precedence — @ConditionalOnMissingBean skips the auto-configured one.

Configuration Properties Reference

Property Description Default

spring.ai.chat.client.tool-search-advisor.enabled

Enable the advisor. Replaces the default ToolCallingAdvisor when true.

false

spring.ai.chat.client.tool-search-advisor.tool-index-type

ToolIndex implementation: regex, lucene, or vector.

regex

spring.ai.chat.client.tool-search-advisor.max-results

Maximum tool references returned per search call. null uses the built-in default.

null

spring.ai.chat.client.tool-search-advisor.system-message-suffix

Custom prompt suffix appended to the system message. null uses the built-in template.

null

spring.ai.chat.client.tool-search-advisor.reference-tool-name-accumulation

When true, accumulate tool names across all search turns; when false, keep only the most recent turn (all parallel calls within it included).

true

spring.ai.chat.client.tool-search-advisor.session-id-key-name

Advisor context key that carries the conversation/session ID.

chat_memory_conversation_id

spring.ai.chat.client.tool-search-advisor.advisor-order

Position of this advisor in the advisor chain.

HIGHEST_PRECEDENCE + 300

spring.ai.chat.client.tool-search-advisor.eviction.lru-max-sessions

Maximum active sessions retained by the LRU eviction strategy.

1000

spring.ai.chat.client.tool-search-advisor.eviction.ttl

TTL for idle sessions. When set, a composite LRU+TTL strategy is used. Accepts a java.time.Duration string (e.g. 30m, 1h).

null

spring.ai.chat.client.tool-search-advisor.lucene.min-score-threshold

Minimum Lucene score for a hit to be included. Applies when tool-index-type=lucene.

0.25

Example Configurations

Lucene with custom threshold and TTL eviction:

spring.ai.chat.client.tool-search-advisor.enabled=true
spring.ai.chat.client.tool-search-advisor.tool-index-type=lucene
spring.ai.chat.client.tool-search-advisor.lucene.min-score-threshold=0.4
spring.ai.chat.client.tool-search-advisor.eviction.ttl=30m

Vector search (requires a VectorStore bean):

spring.ai.chat.client.tool-search-advisor.enabled=true
spring.ai.chat.client.tool-search-advisor.tool-index-type=vector

Custom session-ID key for a multi-tenant deployment:

spring.ai.chat.client.tool-search-advisor.enabled=true
spring.ai.chat.client.tool-search-advisor.tool-index-type=vector
spring.ai.chat.client.tool-search-advisor.session-id-key-name=tenantId

See Also