Package org.springframework.ai.chat.cache.semantic

Class SemanticCacheAdvisor

java.lang.Object
org.springframework.ai.chat.cache.semantic.SemanticCacheAdvisor
All Implemented Interfaces:
Advisor, BaseAdvisor, BaseChatMemoryAdvisor, CallAdvisor, StreamAdvisor, org.springframework.core.Ordered
public class SemanticCacheAdvisor extends Object implements BaseChatMemoryAdvisor
An advisor implementation that provides semantic caching capabilities for chat responses. This advisor intercepts chat requests and checks for semantically similar cached responses before allowing the request to proceed to the model.

This advisor implements BaseChatMemoryAdvisor but overrides both adviseCall(org.springframework.ai.chat.client.ChatClientRequest, org.springframework.ai.chat.client.advisor.api.CallAdvisorChain) and adviseStream(org.springframework.ai.chat.client.ChatClientRequest, org.springframework.ai.chat.client.advisor.api.StreamAdvisorChain) to provide custom caching logic that doesn't fit the standard before/after pattern.

Key features:

  • Semantic similarity based caching of responses
  • Support for both synchronous and streaming chat operations
  • Configurable execution order in the advisor chain
Author:
Brian Sam-Bodden

  • Constructor Details

    • SemanticCacheAdvisor

      public SemanticCacheAdvisor(SemanticCache cache)
      Creates a new semantic cache advisor with default order and scheduler.
      Parameters:
      cache - The semantic cache implementation to use

    • SemanticCacheAdvisor

      public SemanticCacheAdvisor(SemanticCache cache, int order)
      Creates a new semantic cache advisor with specified order and default scheduler.
      Parameters:
      cache - The semantic cache implementation to use
      order - The order of this advisor in the chain

    • SemanticCacheAdvisor

      public SemanticCacheAdvisor(SemanticCache cache, int order, reactor.core.scheduler.Scheduler scheduler)
      Creates a new semantic cache advisor with specified order and scheduler.
      Parameters:
      cache - The semantic cache implementation to use
      order - The order of this advisor in the chain
      scheduler - The scheduler for async operations

  • Method Details

    • getOrder

      public int getOrder()
      Specified by:
      getOrder in interface org.springframework.core.Ordered

    • getScheduler

      public reactor.core.scheduler.Scheduler getScheduler()
      Description copied from interface: BaseAdvisor
      Scheduler used for processing the advisor logic when streaming.
      Specified by:
      getScheduler in interface BaseAdvisor

    • adviseCall

      public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain)
      Handles synchronous chat requests by checking the cache before proceeding. If a semantically similar response is found in the cache, it is returned immediately. Otherwise, the request proceeds through the chain and the response is cached.
      Specified by:
      adviseCall in interface BaseAdvisor
      Specified by:
      adviseCall in interface CallAdvisor
      Parameters:
      request - The chat client request to process
      chain - The advisor chain to continue processing if needed
      Returns:
      The response, either from cache or from the model

    • adviseStream

      public reactor.core.publisher.Flux<ChatClientResponse> adviseStream(ChatClientRequest request, StreamAdvisorChain chain)
      Handles streaming chat requests by checking the cache before proceeding. If a semantically similar response is found in the cache, it is returned as a single item flux. Otherwise, the request proceeds through the chain and the final response is cached.
      Specified by:
      adviseStream in interface BaseAdvisor
      Specified by:
      adviseStream in interface StreamAdvisor
      Parameters:
      request - The chat client request to process
      chain - The advisor chain to continue processing if needed
      Returns:
      A Flux of responses, either from cache or from the model

    • before

      public ChatClientRequest before(ChatClientRequest request, AdvisorChain advisorChain)
      Not used for semantic cache advisor since we override adviseCall/adviseStream.
      Specified by:
      before in interface BaseAdvisor

    • after

      public ChatClientResponse after(ChatClientResponse response, AdvisorChain advisorChain)
      Not used for semantic cache advisor since we override adviseCall/adviseStream.
      Specified by:
      after in interface BaseAdvisor

    • builder

      public static SemanticCacheAdvisor.Builder builder()
      Creates a new builder for constructing SemanticCacheAdvisor instances.
      Returns:
      A new builder instance