Bedrock Anthropic 3

Anthropic Claude is a family of foundational AI models that can be used in a variety of applications.

The Claude model has the following high level features

  • 200k Token Context Window: Claude boasts a generous token capacity of 200,000, making it ideal for handling extensive information in applications like technical documentation, codebase, and literary works.

  • Supported Tasks: Claude’s versatility spans tasks such as summarization, Q&A, trend forecasting, and document comparisons, enabling a wide range of applications from dialogues to content generation.

  • AI Safety Features: Built on Anthropic’s safety research, Claude prioritizes helpfulness, honesty, and harmlessness in its interactions, reducing brand risk and ensuring responsible AI behavior.

The AWS Bedrock Anthropic Model Page and Amazon Bedrock User Guide contains detailed information on how to use the AWS hosted model.

Anthropic’s Claude 2 and 3 models are also available directly on the Anthropic’s own cloud platform. Spring AI provides dedicated Anthropic Claude client to access it.

Prerequisites

Refer to the Spring AI documentation on Amazon Bedrock for setting up API access.

Add Repositories and BOM

Spring AI artifacts are published in Spring Milestone and Snapshot repositories. Refer to the Repositories section to add these repositories to your build system.

To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the Dependency Management section to add the Spring AI BOM to your build system.

Auto-configuration

Add the spring-ai-bedrock-ai-spring-boot-starter dependency to your project’s Maven pom.xml file:

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-bedrock-ai-spring-boot-starter</artifactId>
</dependency>

or to your Gradle build.gradle build file.

dependencies {
    implementation 'org.springframework.ai:spring-ai-bedrock-ai-spring-boot-starter'
}
Refer to the Dependency Management section to add the Spring AI BOM to your build file.

Enable Anthropic Chat

By default the Anthropic model is disabled. To enable it set the spring.ai.bedrock.anthropic3.chat.enabled property to true. Exporting environment variable is one way to set this configuration property:

export SPRING_AI_BEDROCK_ANTHROPIC3_CHAT_ENABLED=true

Chat Properties

The prefix spring.ai.bedrock.aws is the property prefix to configure the connection to AWS Bedrock.

Property Description Default

spring.ai.bedrock.aws.region

AWS region to use.

us-east-1

spring.ai.bedrock.aws.timeout

AWS timeout to use.

5m

spring.ai.bedrock.aws.access-key

AWS access key.

-

spring.ai.bedrock.aws.secret-key

AWS secret key.

-

The prefix spring.ai.bedrock.anthropic3.chat is the property prefix that configures the chat model implementation for Claude.

Property Description Default

spring.ai.bedrock.anthropic3.chat.enabled

Enable Bedrock Anthropic chat model. Disabled by default

false

spring.ai.bedrock.anthropic3.chat.model

The model id to use. Supports the anthropic.claude-3-sonnet-20240229-v1:0,anthropic.claude-3-haiku-20240307-v1:0 and the legacy anthropic.claude-v2, anthropic.claude-v2:1 and anthropic.claude-instant-v1 models for both synchronous and streaming responses.

anthropic.claude-3-sonnet-20240229-v1:0

spring.ai.bedrock.anthropic3.chat.options.temperature

Controls the randomness of the output. Values can range over [0.0,1.0]

0.8

spring.ai.bedrock.anthropic3.chat.options.top-p

The maximum cumulative probability of tokens to consider when sampling.

AWS Bedrock default

spring.ai.bedrock.anthropic3.chat.options.top-k

Specify the number of token choices the generative uses to generate the next token.

AWS Bedrock default

spring.ai.bedrock.anthropic3.chat.options.stop-sequences

Configure up to four sequences that the generative recognizes. After a stop sequence, the generative stops generating further tokens. The returned text doesn’t contain the stop sequence.

10

spring.ai.bedrock.anthropic3.chat.options.anthropic-version

The version of the generative to use.

bedrock-2023-05-31

spring.ai.bedrock.anthropic3.chat.options.max-tokens

Specify the maximum number of tokens to use in the generated response. Note that the models may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. We recommend a limit of 4,000 tokens for optimal performance.

500

Look at the AnthropicChatModel for other model IDs. Supported values are: anthropic.claude-instant-v1, anthropic.claude-v2 and anthropic.claude-v2:1. Model ID values can also be found in the AWS Bedrock documentation for base model IDs.

All properties prefixed with spring.ai.bedrock.anthropic3.chat.options can be overridden at runtime by adding a request specific Runtime Options to the Prompt call.

Runtime Options

The Anthropic3ChatOptions.java provides model configurations, such as temperature, topK, topP, etc.

On start-up, the default options can be configured with the BedrockAnthropicChatModel(api, options) constructor or the spring.ai.bedrock.anthropic3.chat.options.* properties.

At run-time you can override the default options by adding new, request specific, options to the Prompt call. For example to override the default temperature for a specific request:

ChatResponse response = chatModel.call(
    new Prompt(
        "Generate the names of 5 famous pirates.",
        Anthropic3ChatOptions.builder()
            .withTemperature(0.4)
        .build()
    ));
In addition to the model specific AnthropicChatOptions you can use a portable ChatOptions instance, created with the ChatOptionsBuilder#builder().

Multimodal

Multimodality refers to a model’s ability to simultaneously understand and process information from various sources, including text, images, audio, and other data formats. This paradigm represents a significant advancement in AI models.

Currently, Anthropic Claude 3 supports the base64 source type for images, and the image/jpeg, image/png, image/gif, and image/webp media types. Check the Vision guide for more information.

Spring AI’s Message interface supports multimodal AI models by introducing the Media type. This type contains data and information about media attachments in messages, using Spring’s org.springframework.util.MimeType and a java.lang.Object for the raw media data.

Below is a simple code example extracted from Anthropic3ChatModelIT.java, demonstrating the combination of user text with an image.

    byte[] imageData = new ClassPathResource("/test.png").getContentAsByteArray();

    var userMessage = new UserMessage("Explain what do you see o this picture?",
            List.of(new Media(MimeTypeUtils.IMAGE_PNG, imageData)));

    ChatResponse response = chatModel.call(new Prompt(List.of(userMessage)));

    assertThat(response.getResult().getOutput().getContent()).contains("bananas", "apple", "basket");

It takes as an input the test.png image:

Multimodal Test Image

along with the text message "Explain what do you see on this picture?", and generates a response something like:

The image shows a close-up view of a wire fruit basket containing several pieces of fruit.
The basket appears to be made of thin metal wires formed into a round shape with an elevated handle.

Inside the basket, there are a few yellow bananas and a couple of red apples or possibly tomatoes.
The vibrant colors of the fruit contrast nicely against the metallic tones of the wire basket.

The shallow depth of field in the photograph puts the focus squarely on the fruit in the foreground, while the basket handle extending upwards is slightly blurred, creating a pleasing bokeh effect in the background.

The composition and lighting give the image a clean, minimalist aesthetic that highlights the natural beauty and freshness of the fruit displayed in this elegant wire basket.

Sample Controller

Create a new Spring Boot project and add the spring-ai-bedrock-ai-spring-boot-starter to your pom (or gradle) dependencies.

Add a application.properties file, under the src/main/resources directory, to enable and configure the Anthropic chat model:

spring.ai.bedrock.aws.region=eu-central-1
spring.ai.bedrock.aws.timeout=1000ms
spring.ai.bedrock.aws.access-key=${AWS_ACCESS_KEY_ID}
spring.ai.bedrock.aws.secret-key=${AWS_SECRET_ACCESS_KEY}

spring.ai.bedrock.anthropic3.chat.enabled=true
spring.ai.bedrock.anthropic3.chat.options.temperature=0.8
spring.ai.bedrock.anthropic3.chat.options.top-k=15
replace the regions, access-key and secret-key with your AWS credentials.

This will create a BedrockAnthropicChatModel implementation that you can inject into your class. Here is an example of a simple @Controller class that uses the chat model for text generations.

@RestController
public class ChatController {

    private final BedrockAnthropic3ChatModel chatModel;

    @Autowired
    public ChatController(BedrockAnthropic3ChatModel chatModel) {
        this.chatModel = chatModel;
    }

    @GetMapping("/ai/generate")
    public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        return Map.of("generation", chatModel.call(message));
    }

    @GetMapping("/ai/generateStream")
	public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        Prompt prompt = new Prompt(new UserMessage(message));
        return chatModel.stream(prompt);
    }
}

Manual Configuration

The BedrockAnthropic3ChatModel implements the ChatModel and StreamingChatModel and uses the Low-level Anthropic3ChatBedrockApi Client to connect to the Bedrock Anthropic service.

Add the spring-ai-bedrock dependency to your project’s Maven pom.xml file:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-bedrock</artifactId>
</dependency>

or to your Gradle build.gradle build file.

dependencies {
    implementation 'org.springframework.ai:spring-ai-bedrock'
}
Refer to the Dependency Management section to add the Spring AI BOM to your build file.

Next, create an BedrockAnthropic3ChatModel and use it for text generations:

Anthropic3ChatBedrockApi anthropicApi =  new Anthropic3ChatBedrockApi(
    AnthropicChatBedrockApi.AnthropicModel.CLAUDE_V3_SONNET.id(),
    EnvironmentVariableCredentialsProvider.create(),
    Region.US_EAST_1.id(),
    new ObjectMapper(),
    Duration.ofMillis(1000L));

BedrockAnthropic3ChatModel chatModel = new BedrockAnthropic3ChatModel(anthropicApi,
    AnthropicChatOptions.builder()
        .withTemperature(0.6f)
        .withTopK(10)
        .withTopP(0.8f)
        .withMaxTokensToSample(100)
        .withAnthropicVersion(AnthropicChatBedrockApi.DEFAULT_ANTHROPIC_VERSION)
        .build());

ChatResponse response = chatModel.call(
    new Prompt("Generate the names of 5 famous pirates."));

// Or with streaming responses
Flux<ChatResponse> response = chatModel.stream(
    new Prompt("Generate the names of 5 famous pirates."));

Low-level Anthropic3ChatBedrockApi Client

The Anthropic3ChatBedrockApi provides is lightweight Java client on top of AWS Bedrock Anthropic Claude models.

Client supports the anthropic.claude-3-opus-20240229-v1:0,anthropic.claude-3-sonnet-20240229-v1:0,anthropic.claude-3-haiku-20240307-v1:0 and the legacy anthropic.claude-v2, anthropic.claude-v2:1 and anthropic.claude-instant-v1 models for both synchronous (e.g. chatCompletion()) and streaming (e.g. chatCompletionStream()) responses.

Here is a simple snippet how to use the api programmatically:

Anthropic3ChatBedrockApi anthropicChatApi = new Anthropic3ChatBedrockApi(
   AnthropicModel.CLAUDE_V2.id(), Region.EU_CENTRAL_1.id(), Duration.ofMillis(1000L));

AnthropicChatRequest request = AnthropicChatRequest
  .builder(String.format(Anthropic3ChatBedrockApi.PROMPT_TEMPLATE, "Name 3 famous pirates"))
  .withTemperature(0.8f)
  .withMaxTokensToSample(300)
  .withTopK(10)
  .build();

// Sync request
AnthropicChatResponse response = anthropicChatApi.chatCompletion(request);

// Streaming request
Flux<AnthropicChatResponse> responseStream = anthropicChatApi.chatCompletionStream(request);
List<AnthropicChatResponse> responses = responseStream.collectList().block();

Follow the Anthropic3ChatBedrockApi.java's JavaDoc for further information.