|
This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.1.0! |
Chat Client API
The ChatClient offers a fluent API for communicating with an AI Model.
It supports both a synchronous and streaming programming model.
|
See the Implementation Notes at the bottom of this document related to the combined use of imperative and reactive programming models in |
The fluent API has methods for building up the constituent parts of a Prompt that is passed to the AI model as input.
The Prompt contains the instructional text to guide the AI model’s output and behavior.
From the API point of view, prompts consist of a collection of messages.
The AI model processes two main types of messages: user messages, which are direct inputs from the user, and system messages, which are generated by the system to guide the conversation.
These messages often contain placeholders that are substituted at runtime based on user input to customize the response of the AI model to the user input.
There are also Prompt options that can be specified, such as the name of the AI Model to use and the temperature setting that controls the randomness or creativity of the generated output.
Implementation Notes
The combined use of imperative and reactive programming models in ChatClient is a unique aspect of the API.
Often an application will be either reactive or imperative, but not both.
-
Providers built on Spring’s
RestClientandWebClient, such as Ollama and Mistral AI, useRestClientfor calls andWebClientfor streaming. When you customize their HTTP client, configure both. Imperative applications that stream with these providers need the reactive stack (e.g.spring-boot-starter-webflux). Their non-streaming calls block, also in reactive applications. -
Providers built on a vendor SDK, such as OpenAI, Anthropic and Google GenAI, use the SDK’s own HTTP client for calls and streaming, and need neither
RestClientnorWebClient. -
Tool calling is imperative: tools run blocking, and in a streaming call they run on Reactor’s bounded elastic scheduler.
-
The built-in advisors perform blocking operations for standards calls, and non-blocking operations for streaming calls. The Reactor Scheduler used for the advisor streaming calls can be configured via the Builder on each Advisor class.