|
This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.1.0! |
OpenAI Chat
Spring AI supports the various AI language models from OpenAI, the company behind ChatGPT, which has been instrumental in sparking interest in AI-driven text generation thanks to its creation of industry-leading text generation models and embeddings.
|
Spring AI uses the official |
Prerequisites
You will need to create an API with OpenAI to access ChatGPT models.
Create an account at OpenAI signup page and generate the token on the API Keys page.
The Spring AI project defines a configuration property named spring.ai.openai.api-key that you should set to the value of the API Key obtained from openai.com.
You can set this configuration property in your application.properties file:
spring.ai.openai.api-key=<your-openai-api-key>
For enhanced security when handling sensitive information like API keys, you can use Spring Expression Language (SpEL) to reference a custom environment variable:
# In application.yml
spring:
ai:
openai:
api-key: ${OPENAI_API_KEY}
# In your environment or .env file
export OPENAI_API_KEY=<your-openai-api-key>
You can also set this configuration programmatically in your application code:
// Retrieve API key from a secure source or environment variable
String apiKey = System.getenv("OPENAI_API_KEY");
Connection Details
The base URL and API key are resolved through a Spring Boot ConnectionDetails bean of type OpenAiConnectionDetails.
By default, the bean is backed by the spring.ai.openai.base-url and spring.ai.openai.api-key properties.
To develop or test against an OpenAI-compatible model server running in a container (for example via Testcontainers or Docker Compose), declare your own OpenAiConnectionDetails bean pointing at the container.
It supersedes those two properties, while the model-level overrides such as spring.ai.openai.chat.base-url and spring.ai.openai.chat.api-key still take precedence.
@TestConfiguration(proxyBeanMethods = false)
class OpenAiContainerConfiguration {
@Bean
OpenAiConnectionDetails openAiConnectionDetails(GenericContainer<?> modelServer) {
return new OpenAiConnectionDetails() {
@Override
public String getBaseUrl() {
return "http://" + modelServer.getHost() + ":" + modelServer.getFirstMappedPort() + "/v1";
}
@Override
public String getApiKey() {
return "";
}
};
}
}
An empty API key disables authentication, which is what most local OpenAI-compatible servers expect.
Add Repositories and BOM
Spring AI artifacts are published in Maven Central and Spring Snapshot repositories. Refer to the Artifact Repositories section to add these repositories to your build system.
To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the Dependency Management section to add the Spring AI BOM to your build system.
Auto-configuration
Spring AI provides Spring Boot auto-configuration for the OpenAI Chat Client.
To enable it add the following dependency to your project’s Maven pom.xml or Gradle build.gradle build files:
-
Maven
-
Gradle
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
dependencies {
implementation 'org.springframework.ai:spring-ai-starter-model-openai'
}
| Refer to the Dependency Management section to add the Spring AI BOM to your build file. |
Chat Properties
Retry Properties
The OpenAI module does not use the spring.ai.retry properties.
The OpenAI Java SDK retries failed requests itself: set spring.ai.openai.max-retries (default 3) for all OpenAI models, or spring.ai.openai.chat.max-retries for this model only.
See Retry for the providers that use the Spring AI retry.
Connection Properties
The prefix spring.ai.openai is used as the property prefix that lets you connect to OpenAI.
| Property | Description | Default |
|---|---|---|
spring.ai.openai.base-url |
The URL to connect to |
|
spring.ai.openai.api-key |
The API Key |
- |
spring.ai.openai.organization-id |
Optionally, you can specify which organization to use for an API request. |
- |
spring.ai.openai.timeout |
Request timeout for the OpenAI client. |
60 sec. |
spring.ai.openai.max-retries |
Maximum number of retries for the OpenAI client. |
3 |
spring.ai.openai.proxy |
Proxy settings for the OpenAI client. |
- |
spring.ai.openai.custom-headers |
Custom HTTP headers to add to OpenAI client requests. |
empty |
spring.ai.openai.connection-pool-metrics-enabled |
Whether to enable connection pool metrics for the underlying OkHttp client. |
false |
| For users that belong to multiple organizations (or are accessing their projects through their legacy user API key), you can optionally specify which organization and project is used for an API request. Usage from these API requests will count as usage for the specified organization and project. |
User-Agent Header
Spring AI automatically sends a User-Agent: spring-ai header with all requests to OpenAI.
This helps OpenAI identify requests originating from Spring AI for analytics and support purposes.
This header is sent automatically and requires no configuration from Spring AI users.
If you are an API provider building an OpenAI-compatible service, you can track Spring AI usage by reading the User-Agent HTTP header from incoming requests on your server.
Configuration Properties
|
Enabling and disabling of the chat auto-configurations are now configured via top level properties with the prefix To enable, spring.ai.model.chat=openai (It is enabled by default) To disable, spring.ai.model.chat=none (or any value which doesn’t match openai) This change is done to allow configuration of multiple models. |
The prefix spring.ai.openai.chat is the property prefix that lets you configure the chat model implementation for OpenAI.
| Property | Description | Default |
|---|---|---|
spring.ai.model.chat |
Enable OpenAI chat model. |
openai |
spring.ai.openai.chat.api |
The OpenAI endpoint that the |
|
spring.ai.openai.chat.base-url |
Optional override for the |
- |
spring.ai.openai.chat.api-key |
Optional override for the |
- |
spring.ai.openai.chat.organization-id |
Optionally, you can specify which organization to use for an API request. |
- |
spring.ai.openai.chat.model |
Name of the OpenAI chat model to use. You can select between models such as: |
|
spring.ai.openai.chat.temperature |
The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify |
- |
spring.ai.openai.chat.frequency-penalty |
Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model’s likelihood to repeat the same line verbatim. |
- |
spring.ai.openai.chat.logit-bias |
Modify the likelihood of specified tokens appearing in the completion. |
- |
spring.ai.openai.chat.logprobs |
Whether to return the log probabilities of the output tokens. |
- |
spring.ai.openai.chat.top-logprobs |
The number of most likely tokens, between 0 and 20, to return at each token position, each with its log probability. Requires |
- |
spring.ai.openai.chat.max-tokens |
The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length. OpenAI deprecated it in favor of |
- |
spring.ai.openai.chat.max-completion-tokens |
An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens. Required for reasoning models, which include the current GPT-5 and GPT-6 models. Mutually exclusive with maxTokens - setting both will result in an API error. |
- |
spring.ai.openai.chat.n |
How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep |
- |
spring.ai.openai.chat.store |
Whether to store the output of this chat completion request for use in our model |
- |
spring.ai.openai.chat.metadata |
Developer-defined tags and values used for filtering completions in the chat completion dashboard |
empty map |
spring.ai.openai.chat.output-modalities |
Output types that you would like the model to generate for this request. Most models are capable of generating text, which is the default.
The |
- |
spring.ai.openai.chat.output-audio |
Audio parameters for the audio generation. Required when audio output is requested with |
- |
spring.ai.openai.chat.presence-penalty |
Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model’s likelihood to talk about new topics. |
- |
spring.ai.openai.chat.response-format.type |
The |
- |
spring.ai.openai.chat.response-format.json-schema |
Response format JSON schema. Applicable only for |
- |
spring.ai.openai.chat.response-format.strict |
Response format JSON schema adherence strictness. Applicable only for |
|
spring.ai.openai.chat.seed |
This feature is in Beta. If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. |
- |
spring.ai.openai.chat.stop |
Up to 4 sequences where the API will stop generating further tokens. |
- |
spring.ai.openai.chat.stream-options.include-usage |
Whether a streaming response reports the token usage. When no |
- |
spring.ai.openai.chat.stream-options.include-obfuscation |
Whether streaming responses carry the obfuscation field that OpenAI adds to normalize payload sizes. |
- |
spring.ai.openai.chat.top-p |
An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with |
- |
spring.ai.openai.chat.tool-choice |
Controls which (if any) function is called by the model. |
- |
spring.ai.openai.chat.user |
A unique identifier representing your end-user, which can help OpenAI to monitor and detect abuse. |
- |
spring.ai.openai.chat.parallel-tool-calls |
Whether to enable parallel function calling during tool use. |
- |
spring.ai.openai.chat.reasoning-effort |
Controls the reasoning effort for models that support it. The value is forwarded to the OpenAI Chat Completions API; use only values supported by the selected model. For models that do not support function tools when reasoning is enabled, set this to |
- |
spring.ai.openai.chat.verbosity |
Constrains how verbose the response is, for models that support it, such as GPT-5: |
- |
spring.ai.openai.chat.prompt-cache-key |
A cache key used by OpenAI to optimize cache hit rates for similar requests. Improves latency and reduces costs. Replaces the deprecated |
- |
spring.ai.openai.chat.custom-headers |
Optional HTTP headers to be added to the chat completion request. To override the |
- |
spring.ai.openai.chat.service-tier |
Specifies the processing type used for serving the request. |
- |
spring.ai.openai.chat.extra-body |
Additional parameters to include in the request. Accepts any key-value pairs that are flattened to the top level of the JSON request. Intended for use with OpenAI-compatible servers (vLLM, Ollama, etc.) that support parameters beyond the standard OpenAI API. The official OpenAI API rejects unknown parameters with a 400 error. See Using Extra Parameters with OpenAI-Compatible Servers for details. |
- |
spring.ai.openai.chat.replay-reasoning-content |
Whether a previous assistant message’s reasoning content is sent back as the |
- |
|
When using a local Docker Model Runner that exposes an OpenAI-compatible API, you may need to append |
|
The GPT-5.2 and GPT-5.4 models, such as |
You can override the common spring.ai.openai.base-url and spring.ai.openai.api-key for the ChatModel and EmbeddingModel implementations.
The spring.ai.openai.chat.base-url and spring.ai.openai.chat.api-key properties, if set, take precedence over the common properties.
This is useful if you want to use different OpenAI accounts for different models and different model endpoints.
|
All properties prefixed with spring.ai.openai.chat can be overridden at runtime by adding request-specific Runtime Options to the Prompt call.
|
Token Limit Parameters: Model-Specific Usage
OpenAI provides two mutually exclusive parameters for controlling token generation limits:
| Parameter | Use Case | Compatible Models |
|---|---|---|
|
Deprecated by OpenAI; not for reasoning models |
Older models, and OpenAI-compatible servers that do not accept |
|
Reasoning models |
The current models, such as gpt-5.4-mini, gpt-5.4 and gpt-6-luna |
| These parameters are mutually exclusive. Setting both will result in an API error from OpenAI. |
Usage Examples
For the current models, which are reasoning models:
ChatResponse response = chatModel.call(
new Prompt(
"Solve this complex math problem step by step: ...",
OpenAiChatOptions.builder()
.model("gpt-5.4-mini")
.maxCompletionTokens(1000) // Use maxCompletionTokens for reasoning models
.build()
));
Use maxTokens only with a model or an OpenAI-compatible server that does not accept max_completion_tokens.
Builder Pattern Validation:
The OpenAiChatOptions builder never keeps both: maxCompletionTokens wins, whichever of the two is set first:
// maxCompletionTokens clears maxTokens and logs a warning
OpenAiChatOptions options = OpenAiChatOptions.builder()
.maxTokens(100)
.maxCompletionTokens(200)
.build();
// maxTokens is ignored, with a warning, once maxCompletionTokens is set
OpenAiChatOptions sameOptions = OpenAiChatOptions.builder()
.maxCompletionTokens(200)
.maxTokens(100)
.build();
// Both: maxTokens = null, maxCompletionTokens = 200
Runtime Options
The OpenAiChatOptions.java class provides model configurations such as the model to use, the temperature, the frequency penalty, etc.
On start-up, the default options can be configured with OpenAiChatModel.builder().options(…) or the spring.ai.openai.chat.* properties.
At run-time, you can override the default options by adding new, request-specific options to the Prompt call.
For example, to override the default model and temperature for a specific request:
ChatResponse response = chatModel.call(
new Prompt(
"Generate the names of 5 famous pirates.",
OpenAiChatOptions.builder()
.model("gpt-5.4-mini")
.build()
));
| In addition to the model specific OpenAiChatOptions you can use a portable ChatOptions instance, created with ChatOptions#builder(). |
Rate Limit Metadata
OpenAI reports your current request and token budgets in rate limit response headers.
Spring AI exposes them through the standard ChatResponseMetadata#getRateLimit() accessor, which returns an OpenAiRateLimit populated from those headers.
ChatResponse response = chatModel.call(prompt);
RateLimit rateLimit = response.getMetadata().getRateLimit();
rateLimit.getRequestsLimit(); // overall request ceiling
rateLimit.getRequestsRemaining(); // requests left in the current window
rateLimit.getRequestsReset(); // time until the request window resets
rateLimit.getTokensLimit(); // overall token ceiling
rateLimit.getTokensRemaining(); // tokens left in the current window
rateLimit.getTokensReset(); // time until the token window resets
If the response carries no rate-limit headers, getRateLimit() returns an EmptyRateLimit instance instead.
Rate-limit metadata is populated on call(Prompt) only.
Reading it on stream(Prompt) would require the SDK’s blocking streaming response, which would hold a worker thread for the whole stream, so stream(Prompt) reports an EmptyRateLimit.
|
Tool Calling
OpenAiChatModel supports tool calling.
The model can request tool execution but does not run tools itself — Spring AI handles execution.
For most applications, use ChatClient with the auto-registered ToolCallingAdvisor — see Tool Calling.
For low-level control where you drive the loop yourself, see ChatModel Tool Calling.
Multimodal
Multimodality refers to a model’s ability to simultaneously understand and process information from various sources, including text, images, audio, and other data formats. OpenAI supports text, vision, and audio input modalities.
Vision
The current OpenAI chat models, such as gpt-5.4-mini, accept image input.
Refer to the Vision guide for more information.
The OpenAI User Message API can incorporate a list of base64-encoded images or image urls with the message.
Spring AI’s Message interface facilitates multimodal AI models by introducing the Media type.
This type encompasses data and details regarding media attachments in messages, utilizing Spring’s org.springframework.util.MimeType and a org.springframework.core.io.Resource for the raw media data.
Below is a code example excerpted from OpenAiChatModelIT.java, illustrating the fusion of user text with an image using the gpt-4o model.
-
MediaPart
-
media(…)
var imageResource = new ClassPathResource("/multimodal.test.png");
var userMessage = UserMessage.builder()
.part(TextPart.of("Explain what do you see on this picture?"))
.part(MediaPart.of(new Media(MimeTypeUtils.IMAGE_PNG, imageResource)))
.build();
ChatResponse response = chatModel.call(new Prompt(userMessage,
OpenAiChatOptions.builder().model("gpt-5.4-mini").build()));
var imageResource = new ClassPathResource("/multimodal.test.png");
var userMessage = UserMessage.builder()
.text("Explain what do you see on this picture?")
.media(new Media(MimeTypeUtils.IMAGE_PNG, imageResource))
.build();
ChatResponse response = chatModel.call(new Prompt(userMessage,
OpenAiChatOptions.builder().model("gpt-5.4-mini").build()));
The content parts of a user message are sent in the order of its message parts, so text and images can interleave.
A message built with UserMessage.builder().part(TextPart.of("Look at ")).part(MediaPart.of(image)).part(TextPart.of("this.")) reaches the model as text, image, text.
Messages built with the constructors or the text(…) and media(…) setters keep the text-then-media order.
or the image URL equivalent:
-
MediaPart
-
media(…)
var userMessage = UserMessage.builder()
.part(TextPart.of("Explain what do you see on this picture?"))
.part(MediaPart.of(new Media(MimeTypeUtils.IMAGE_PNG,
URI.create("https://docs.spring.io/spring-ai/reference/_images/multimodal.test.png"))))
.build();
ChatResponse response = chatModel.call(new Prompt(userMessage,
OpenAiChatOptions.builder().model("gpt-5.4-mini").build()));
var userMessage = UserMessage.builder()
.text("Explain what do you see on this picture?")
.media(new Media(MimeTypeUtils.IMAGE_PNG,
URI.create("https://docs.spring.io/spring-ai/reference/_images/multimodal.test.png")))
.build();
ChatResponse response = chatModel.call(new Prompt(userMessage,
OpenAiChatOptions.builder().model("gpt-5.4-mini").build()));
| You can pass multiple images as well. |
The example shows a model taking as an input the multimodal.test.png image:
along with the text message "Explain what do you see on this picture?", and generating a response like this:
This is an image of a fruit bowl with a simple design. The bowl is made of metal with curved wire edges that create an open structure, allowing the fruit to be visible from all angles. Inside the bowl, there are two yellow bananas resting on top of what appears to be a red apple. The bananas are slightly overripe, as indicated by the brown spots on their peels. The bowl has a metal ring at the top, likely to serve as a handle for carrying. The bowl is placed on a flat surface with a neutral-colored background that provides a clear view of the fruit inside.
Audio
OpenAI models that offer input audio multimodal support include gpt-audio-1.5.
Refer to the Audio guide for more information.
The OpenAI User Message API can incorporate a list of base64-encoded audio files with the message.
Spring AI’s Message interface facilitates multimodal AI models by introducing the Media type.
This type encompasses data and details regarding media attachments in messages, utilizing Spring’s org.springframework.util.MimeType and a org.springframework.core.io.Resource for the raw media data.
Currently, OpenAI support only the following media types: audio/mp3 and audio/wav.
Below is a code example excerpted from OpenAiChatModelIT.java, illustrating the fusion of user text with an audio file using the gpt-audio-1.5 model.
-
MediaPart
-
media(…)
var audioResource = new ClassPathResource("speech1.mp3");
var userMessage = UserMessage.builder()
.part(TextPart.of("What is this recording about?"))
.part(MediaPart.of(new Media(MimeTypeUtils.parseMimeType("audio/mp3"), audioResource)))
.build();
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
OpenAiChatOptions.builder().model("gpt-audio-1.5").build()));
var audioResource = new ClassPathResource("speech1.mp3");
var userMessage = UserMessage.builder()
.text("What is this recording about?")
.media(new Media(MimeTypeUtils.parseMimeType("audio/mp3"), audioResource))
.build();
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
OpenAiChatOptions.builder().model("gpt-audio-1.5").build()));
| You can pass multiple audio files as well. |
Output Audio
OpenAI models that offer input audio multimodal support include gpt-audio-1.5.
Refer to the Audio guide for more information.
The OpenAI Assistant Message API can contain a list of base64-encoded audio files with the message.
Spring AI’s Message interface facilitates multimodal AI models by introducing the Media type.
This type encompasses data and details regarding media attachments in messages, utilizing Spring’s org.springframework.util.MimeType and a org.springframework.core.io.Resource for the raw media data.
Currently, OpenAI support only the following audio types: audio/mp3 and audio/wav.
Below is a code example, illustrating the response of user text along with an audio byte array, using the gpt-audio-1.5 model:
var userMessage = new UserMessage("Tell me joke about Spring Framework");
ChatResponse response = chatModel.call(new Prompt(List.of(userMessage),
OpenAiChatOptions.builder()
.model("gpt-audio-1.5")
.outputModalities(List.of("text", "audio"))
.outputAudio(new AudioParameters(Voice.ALLOY, AudioResponseFormat.WAV))
.build()));
String text = response.getResult().getOutput().getText(); // audio transcript
byte[] waveAudio = response.getResult().getOutput().getMedia().get(0).getDataAsByteArray(); // audio data
You have to specify an audio modality in the OpenAiChatOptions to generate audio output.
The AudioParameters class provides the voice and audio format for the audio output.
Message Parts
Each choice of a response is a Generation whose AssistantMessage holds the choice as ordered message parts:
| Chat Completions field | Message part |
|---|---|
|
|
|
|
a |
|
|
|
The parts follow that order: reasoning, text, tool calls, audio.
A choice without content has no TextPart, but getText() still returns an empty string, as it always has.
When an assistant message is sent back, its parts are folded into a single Chat Completions message, because the API has no way to interleave them:
-
the text parts are joined into
content; -
the reasoning parts are joined into
reasoning_content, unless another provider signed some of them (for example Anthropic thinking), in which case no reasoning is sent, or unless the replay is turned off (see Replaying Reasoning Content in Multi-Turn Conversations); -
the
ToolCallPart`s become `tool_calls, with their extra fields restored; -
media and any other part are not sent.
Reasoning parts without a payload are replayable to any provider, so reasoning that another chat model produced without a signature is sent as reasoning_content when the history is replayed to OpenAiChatModel.
An example is the reasoning text of an OpenAiResponsesChatModel turn whose reasoning item carries no encrypted content, as an open-weight model such as gpt-oss behind a Responses-compatible server returns it.
If the server you target rejects that field, set replayReasoningContent to false (see Replaying Reasoning Content in Multi-Turn Conversations).
|
When streaming, the reasoning and text deltas of each chunk are partial parts stamped with a synthesized index, so that MessageAggregator rebuilds one ReasoningPart and one TextPart.
While a tool call streams, each of its argument deltas arrives right away as a partial ToolCallPart, which is only for display and is not a tool call (see Streaming Tool Call Arguments).
Once the model has finished its tool calls, one more chunk carries them as complete ToolCallPart`s, at the same index as their deltas, and only those are returned by `getToolCalls().
Structured Outputs
OpenAI provides custom Structured Outputs APIs that ensure your model generates responses conforming strictly to your provided JSON Schema.
In addition to the existing Spring AI model-agnostic Structured Output, these APIs offer enhanced control and precision.
| Currently, OpenAI supports a subset of the JSON Schema language format. |
Configuration
Spring AI allows you to configure your response format either programmatically using the OpenAiChatOptions builder or through application properties.
Using the Chat Options Builder
You can set the response format programmatically with the OpenAiChatOptions builder as shown below:
String jsonSchema = """
{
"type": "object",
"properties": {
"steps": {
"type": "array",
"items": {
"type": "object",
"properties": {
"explanation": { "type": "string" },
"output": { "type": "string" }
},
"required": ["explanation", "output"],
"additionalProperties": false
}
},
"final_answer": { "type": "string" }
},
"required": ["steps", "final_answer"],
"additionalProperties": false
}
""";
Prompt prompt = new Prompt("how can I solve 8x + 7 = -23",
OpenAiChatOptions.builder()
.model("gpt-5.4-mini")
.responseFormat(ResponseFormat.builder()
.type(ResponseFormat.Type.JSON_SCHEMA)
.jsonSchema(jsonSchema)
.build())
.build());
ChatResponse response = openAiChatModel.call(prompt);
| Adhere to the OpenAI subset of the JSON Schema language format. |
Integrating with BeanOutputConverter Utilities
You can leverage existing BeanOutputConverter utilities to automatically generate the JSON Schema from your domain objects and later convert the structured response into domain-specific instances:
-
Java
-
Kotlin
record MathReasoning(
@JsonProperty(required = true, value = "steps") Steps steps,
@JsonProperty(required = true, value = "final_answer") String finalAnswer) {
record Steps(
@JsonProperty(required = true, value = "items") Items[] items) {
record Items(
@JsonProperty(required = true, value = "explanation") String explanation,
@JsonProperty(required = true, value = "output") String output) {
}
}
}
var outputConverter = new BeanOutputConverter<>(MathReasoning.class);
var jsonSchema = outputConverter.getJsonSchema();
Prompt prompt = new Prompt("how can I solve 8x + 7 = -23",
OpenAiChatOptions.builder()
.model("gpt-5.4-mini")
.responseFormat(ResponseFormat.builder()
.type(ResponseFormat.Type.JSON_SCHEMA)
.jsonSchema(jsonSchema)
.build())
.build());
ChatResponse response = openAiChatModel.call(prompt);
String content = response.getResult().getOutput().getText();
MathReasoning mathReasoning = outputConverter.convert(content);
data class MathReasoning(
val steps: Steps,
@get:JsonProperty(value = "final_answer") val finalAnswer: String) {
data class Steps(val items: Array<Items>) {
data class Items(
val explanation: String,
val output: String)
}
}
val outputConverter = BeanOutputConverter(MathReasoning::class.java)
val jsonSchema = outputConverter.jsonSchema;
val prompt = Prompt("how can I solve 8x + 7 = -23",
OpenAiChatOptions.builder()
.model("gpt-5.4-mini")
.responseFormat(ResponseFormat.builder()
.type(ResponseFormat.Type.JSON_SCHEMA)
.jsonSchema(jsonSchema)
.build())
.build())
val response = openAiChatModel.call(prompt)
val content = response.getResult().getOutput().getText()
val mathReasoning = outputConverter.convert(content)
Although this is optional for JSON Schema, OpenAI mandates required fields for the structured response to function correctly.
Kotlin reflection is used to infer which property are required or not based on the nullability of types and default values of parameters, so for most use case @get:JsonProperty(required = true) is not needed.
@get:JsonProperty(value = "custom_name") can be useful to customize the property name.
Make sure to generate the annotation on the related getters with this @get: syntax, see related documentation.
|
Configuring via Application Properties
Alternatively, when using the OpenAI auto-configuration, you can configure the desired response format through the following application properties:
spring.ai.openai.api-key=YOUR_API_KEY
spring.ai.openai.chat.model=gpt-5.4-mini
spring.ai.openai.chat.response-format.type=JSON_SCHEMA
spring.ai.openai.chat.response-format.json-schema={"type":"object","properties":{"steps":{"type":"array","items":{"type":"object","properties":{"explanation":{"type":"string"},"output":{"type":"string"}},"required":["explanation","output"],"additionalProperties":false}},"final_answer":{"type":"string"}},"required":["steps","final_answer"],"additionalProperties":false}
spring.ai.openai.chat.response-format.strict=true
Sample Controller
Create a new Spring Boot project and add the spring-ai-starter-model-openai to your pom (or gradle) dependencies.
Add an application.properties file under the src/main/resources directory to enable and configure the OpenAi chat model:
spring.ai.openai.api-key=YOUR_API_KEY
spring.ai.openai.chat.model=gpt-5.4-mini
Replace the api-key with your OpenAI credentials.
|
This will create an OpenAiChatModel implementation that you can inject into your classes.
Here is an example of a simple @RestController class that uses the chat model for text generations.
@RestController
public class ChatController {
private final OpenAiChatModel chatModel;
@Autowired
public ChatController(OpenAiChatModel chatModel) {
this.chatModel = chatModel;
}
@GetMapping("/ai/generate")
public Map<String,String> generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of("generation", this.chatModel.call(message));
}
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return this.chatModel.stream(prompt);
}
}
Manual Configuration
The OpenAiChatModel implements the ChatModel and StreamingChatModel and uses the official OpenAI Java SDK to connect to the OpenAI service.
Add the spring-ai-openai dependency to your project’s Maven pom.xml file:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-openai</artifactId>
</dependency>
or to your Gradle build.gradle build file.
dependencies {
implementation 'org.springframework.ai:spring-ai-openai'
}
| Refer to the Dependency Management section to add the Spring AI BOM to your build file. |
Next, create an OpenAiChatModel and use it for text generations:
var chatModel = OpenAiChatModel.builder()
.options(OpenAiChatOptions.builder()
.apiKey(System.getenv("OPENAI_API_KEY"))
.model("gpt-5.4-mini")
.maxCompletionTokens(200)
.build())
.build();
ChatResponse response = this.chatModel.call(
new Prompt("Generate the names of 5 famous pirates."));
// Or with streaming responses
Flux<ChatResponse> response = this.chatModel.stream(
new Prompt("Generate the names of 5 famous pirates."));
The OpenAiChatOptions provides the configuration information for the chat requests.
The OpenAiChatOptions.Builder is a fluent options-builder for the chat config.
API Key Management
Spring AI provides flexible API key management through the ApiKey interface and its implementations.
The default implementation, SimpleApiKey, is suitable for most use cases, but you can also create custom implementations for more complex scenarios.
Default Configuration
By default, Spring Boot auto-configuration will create an API key bean using the spring.ai.openai.api-key property:
spring.ai.openai.api-key=your-api-key-here
Custom API Key Configuration
You can create a custom instance of OpenAiChatModel with your own ApiKey implementation using the builder pattern:
ApiKey customApiKey = new ApiKey() {
@Override
public String getValue() {
// Custom logic to retrieve API key
return "your-api-key-here";
}
};
// Create a chat model with the custom API key
OpenAiChatModel chatModel = OpenAiChatModel.builder()
.options(OpenAiChatOptions.builder()
.apiKey(customApiKey.getValue())
.build())
.build();
// Build the ChatClient using the custom chat model
ChatClient openAiChatClient = ChatClient.builder(chatModel).build();
This is useful when you need to:
-
Retrieve the API key from a secure key store
-
Rotate API keys dynamically
-
Implement custom API key selection logic
Observability
Spring AI emits Micrometer observations at two layers for every OpenAI call:
-
Chat-model layer. A
gen_ai.client.operationobservation wraps every call toOpenAiChatModel.call(…)or.stream(…), carrying request parameters, response metadata, and token usage. See the observability reference for the full set of tags and metrics. -
HTTP layer. An
okhttp.requestsobservation wraps every HTTP attempt to the OpenAI API, carrying HTTP method, URI, status code, and exception tags. Each request also propagatestraceparenton the wire to any downstream services (AI gateways, OpenAI-compatible inference servers, proxies).
In a Spring Boot application with spring-boot-starter-actuator and a tracing bridge (micrometer-tracing-bridge-otel or micrometer-tracing-bridge-brave), both layers are wired automatically — no opt-in required.
For non-Boot applications, pass an ObservationRegistry to OpenAiChatModel.builder():
OpenAiChatModel chatModel = OpenAiChatModel.builder()
.options(...)
.observationRegistry(observationRegistry)
.build();
Connection-pool metrics (opt-in)
Spring AI can additionally bind OkHttp connection-pool gauges (okhttp.pool.connection.count with state=active|idle and client.kind=sync|async) to the application’s MeterRegistry.
These are secondary telemetry — useful for capacity tuning but not required for tracing or per-request latency — so they’re disabled by default.
To enable them in a Spring Boot application:
spring.ai.openai.chat.connection-pool-metrics-enabled=true
For non-Boot applications, pass a MeterRegistry directly to the builder; the gauges bind whenever a registry is supplied:
OpenAiChatModel chatModel = OpenAiChatModel.builder()
.options(...)
.observationRegistry(observationRegistry)
.meterRegistry(meterRegistry)
.build();
HTTP dispatcher executor (advanced)
By default Spring AI manages the OkHttp dispatcher’s executor — an unbounded pool of platform threads, replicating the SDK’s stock behavior.
For workloads with high HTTP concurrency, or to take advantage of Java 21+ virtual threads, you can supply your own ExecutorService:
OpenAiChatModel chatModel = OpenAiChatModel.builder()
.options(...)
.httpClientBuilderCustomizer(httpClient -> httpClient
.dispatcherExecutorService(Executors.newVirtualThreadPerTaskExecutor()))
.build();
The same executor backs both the sync and async (streaming) clients.
When you supply your own executor, you own its lifecycle — Spring AI will never call shutdown() on it.
When omitted, the internal executor is created and cleaned up automatically.
|
Streaming HTTP observations are not parented under the chat-model observation For synchronous calls, the For streaming calls, the If you need to correlate streaming HTTP spans with their parent chat-model span in your tracing UI, filter by span name |
Customizing the HTTP Client
Spring AI uses the official openai-java SDK under the hood and configures its HTTP transport with a custom OkHttp client built by SpringAiOpenAiHttpClient.Builder.
You can intercept that builder before the underlying OkHttpClient is created by exposing one or more OpenAiHttpClientBuilderCustomizer beans.
Each customizer receives the same builder used by every OpenAI model (chat, embedding, image, audio, moderation), so the customization applies uniformly.
@FunctionalInterface
public interface OpenAiHttpClientBuilderCustomizer {
void customize(SpringAiOpenAiHttpClient.Builder builder);
}
Typical use cases include:
-
registering OkHttp
Interceptorinstances (authentication, propagation headers, custom logging); -
swapping the dispatcher
ExecutorService(for example, to route async I/O through virtual threads); -
configuring proxy, SSL, hostname verification, or the connection-pool sizing exposed by the builder.
When several customizers are present, they are applied in @Order / Ordered order, after Spring AI’s own defaults, so user code wins.
The same hook is available when wiring a model manually via the OpenAi*Model.Builder:
var chatModel = OpenAiChatModel.builder()
.options(OpenAiChatOptions.builder().model("gpt-4o").build())
.httpClientBuilderCustomizer(myCustomizer)
.build();