This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.0.2!

Ollama Decision Models

With Ollama, you can run decision models on your own machine and use them through the Decision Models API. A decision model picks one result from a known set of possible results, such as yes or no, one of a few categories, or a level on a scale, and tells you how likely each result is.

OllamaDecisionModel uses the Ollama decision endpoint, /v1/systemone, through the TypeSafe Java SDK. This endpoint is separate from chat, and only models with the decision capability can use it.

Prerequisites

You need access to an Ollama instance, for example:

Download and install Ollama on your machine.
Run Ollama with Testcontainers.

The decision endpoint needs a recent version of Ollama. We recommend Ollama 0.40.4 or later.

Pull the decision model that you want to use from the Ollama model library:

ollama pull d1-omni:600m

To check that a model supports decisions, run ollama show <model-name>: its capabilities include decision.

Spring AI can also pull the model for you at startup, as described in Auto-pulling Models.

Auto-configuration

Spring AI provides Spring Boot auto-configuration for the Ollama decision model. To enable it, add the following dependency to your Maven pom.xml or Gradle build.gradle build files:

Maven
Gradle

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>

dependencies {
    implementation 'org.springframework.ai:spring-ai-starter-model-ollama'
}

Refer to the Dependency Management section to add the Spring AI BOM to your build file.

The starter includes the TypeSafe Java SDK that the decision model needs. If your application doesn’t use the decision model, turn off its auto-configuration with spring.ai.model.decision=none, as described in Decision Properties.

Connection Properties

The prefix spring.ai.ollama is the property prefix to configure the connection to Ollama.

Property

Description

Default

spring.ai.ollama.base-url

The base URL of the Ollama server.

http://localhost:11434

The decision model sends its requests to the same Ollama server as the chat and embedding models. Ollama needs no API key, so the auto-configuration gives the TypeSafe SDK a placeholder key. The TYPESAFE_API_KEY and TYPESAFE_BASE_URL environment variables of the SDK are not used.

The following properties configure the Ollama initialization and the automatic pulling of models.

Property

Description

Default

spring.ai.ollama.init.pull-model-strategy

Whether to pull models at startup, and how.

never

spring.ai.ollama.init.timeout

How long to wait for a model to be pulled.

5m

spring.ai.ollama.init.max-retries

The maximum number of retries for pulling a model.

0

spring.ai.ollama.init.decision.include

Whether to include decision models in the initialization.

true

spring.ai.ollama.init.decision.additional-models

More decision models to pull, besides the one set with spring.ai.ollama.decision.model.

[]

Decision Properties

The spring.ai.model.decision property enables and disables the decision auto-configuration.

To enable it, set spring.ai.model.decision=ollama (this is the default).

To disable it, set spring.ai.model.decision=none, or any value other than ollama.

The prefix spring.ai.ollama.decision is the property prefix to configure the Ollama decision model.

Property

Description

Default

spring.ai.model.decision

Enables the Ollama decision model.

ollama

spring.ai.ollama.decision.model

The name of the decision model to use.

d1-omni:600m

You can change the spring.ai.ollama.decision.model property for one call, with the Runtime Options of the DecisionPrompt.

Runtime Options

OllamaDecisionOptions holds the settings of a decision call. At startup, the model uses the settings of the spring.ai.ollama.decision properties. You can change them for one call by setting options on the prompt:

DecisionResponse response = ollamaDecisionModel.call(DecisionPrompt.builder()
    .text("My new headphones arrived yesterday, but the left side doesn't work. Can I get my money back?")
    .question("defect", YesNoQuestion.builder()
        .text("Does the customer report a defect?")
        .whenYes("Part of the item does not work as it should")
        .whenNo("The item works, and the message is about something else")
        .build())
    .question("topic", ChoiceQuestion.builder()
        .text("What is the message about?")
        .option("delivery", "Where an order is, or when it will arrive")
        .option("product", "A problem with an item the customer received")
        .option("account", "Login, address or payment details")
        .option("other", "None of the above")
        .build())
    .question("satisfaction", ScoreQuestion.of("How satisfied is the customer?", "unhappy", "neutral", "happy"))
    .options(OllamaDecisionOptions.builder()
        .model("d1-omni:2b")
        .build())
    .build());

boolean defect = response.getYesNo("defect").isYes();
String topic = response.getChoice("topic").getValue();
int satisfaction = response.getScore("satisfaction").getNearestLevel();

The model name is the only setting, so the portable DecisionOptions work too.

Small decision models, such as d1-omni:600m, depend a lot on the descriptions of the options. Describe each option by the situation it covers, as the topic options do.

Auto-pulling Models

Spring AI can pull models automatically when they are not available in your Ollama instance. The strategies, the timeout and the maximum number of retries are the same as for the chat models.

You can pull more decision models at startup, for example models that you choose at runtime:

spring:
  ai:
    ollama:
      init:
        pull-model-strategy: when_missing
        decision:
          additional-models:
            - d1-omni:2b

To apply the pulling strategy only to other types of models, exclude decision models from the initialization:

spring:
  ai:
    ollama:
      init:
        pull-model-strategy: always
        decision:
          include: false

Downloading a model can take a long time, so we don’t recommend automatic pulling in production. Instead, download the models that you need in advance.

Supported Features

Feature Supported Notes

Feature	Supported	Notes
Text input	Yes
Text with images	No	Ollama decides about text only, so an image in the input makes the call fail.
Fields input	Yes	The model can answer differently when the order of the fields changes, so use a `LinkedHashMap`.
Conversation input	Yes	Tool calls and tool results are sent, reasoning is not.
Probabilities	Yes	For all three kinds of questions.
Confidence	Yes	For choice and score answers. A confidence of `0` can mean that the server reported none.
Refusals	No
Response id	No	`response.getMetadata().getId()` is `null`.

Text input

Yes

Text with images

Ollama decides about text only, so an image in the input makes the call fail.

Fields input

Yes

The model can answer differently when the order of the fields changes, so use a LinkedHashMap.

Conversation input

Yes

Tool calls and tool results are sent, reasoning is not.

Probabilities

Yes

For all three kinds of questions.

Confidence

Yes

For choice and score answers. A confidence of 0 can mean that the server reported none.

Refusals

Response id

response.getMetadata().getId() is null.

Request and Response Mapping

Ollama serves the System One protocol, and OllamaDecisionModel maps requests and responses in the same way as the System One integration. See Request Mapping and Response Mapping on the System One page.

Observability

Spring AI observes the calls as decision operations, with ollama as the gen_ai.system key. See Decision Model observability.

Sample Controller

The starter creates a DecisionModel bean that you can inject into your classes. Here is a simple @RestController that uses it to sort customer messages:

@RestController
public class DecisionController {

    private final DecisionModel decisionModel;

    public DecisionController(DecisionModel decisionModel) {
        this.decisionModel = decisionModel;
    }

    @GetMapping("/ai/decision")
    public Map<String, Object> triage(@RequestParam(value = "message",
            defaultValue = "Where is my order? I ordered it two weeks ago.") String message) {
        DecisionResponse response = this.decisionModel.call(DecisionPrompt.builder()
            .text(message)
            .question("defect", YesNoQuestion.of("Does the customer report a defect?"))
            .question("topic", ChoiceQuestion.of("What is the message about?",
                    "delivery", "product", "account", "other"))
            .build());
        return Map.of("defect", response.getYesNo("defect").isYes(),
                "topic", response.getChoice("topic").getValue());
    }

}

Manual Configuration

If you don’t use Spring Boot, you can configure the OllamaDecisionModel yourself. Add the spring-ai-ollama dependency to your Maven pom.xml or Gradle build.gradle build files:

Maven
Gradle

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-ollama</artifactId>
</dependency>

dependencies {
    implementation 'org.springframework.ai:spring-ai-ollama'
}

Refer to the Dependency Management section to add the Spring AI BOM to your build file.

The model needs two clients for the same Ollama server:

an OllamaApi, to pull models;
a TypeSafeClient, to send the decisions.

Ollama needs no API key, but the TypeSafeClient requires one, so give it any value that is not empty.

OllamaApi ollamaApi = OllamaApi.builder().build();

TypeSafeClient systemOneClient = TypeSafeClient.builder()
    .baseUrl("http://localhost:11434")
    .apiKey("ollama")
    .build();

OllamaDecisionModel decisionModel = OllamaDecisionModel.builder()
    .ollamaApi(ollamaApi)
    .systemOneClient(systemOneClient)
    .decisionOptions(OllamaDecisionOptions.builder()
        .model("d1-omni:600m")
        .build())
    .build();

DecisionResponse response = decisionModel.call("Where is my order? I ordered it two weeks ago.",
        Map.of("topic", ChoiceQuestion.of("What is the message about?",
                "delivery", "product", "account", "other")));

When the options set no model, the model uses d1-omni:600m.

Example Code

The OllamaDecisionModelIT test has more examples of how to use the Ollama decision model.