Bedrock Anthropic 3
Anthropic Claude is a family of foundational AI models that can be used in a variety of applications.
The Claude model has the following high level features
-
200k Token Context Window: Claude boasts a generous token capacity of 200,000, making it ideal for handling extensive information in applications like technical documentation, codebase, and literary works.
-
Supported Tasks: Claude’s versatility spans tasks such as summarization, Q&A, trend forecasting, and document comparisons, enabling a wide range of applications from dialogues to content generation.
-
AI Safety Features: Built on Anthropic’s safety research, Claude prioritizes helpfulness, honesty, and harmlessness in its interactions, reducing brand risk and ensuring responsible AI behavior.
The AWS Bedrock Anthropic Model Page and Amazon Bedrock User Guide contains detailed information on how to use the AWS hosted model.
Anthropic’s Claude 2 and 3 models are also available directly on the Anthropic’s own cloud platform. Spring AI provides dedicated Anthropic Claude client to access it. |
Prerequisites
Refer to the Spring AI documentation on Amazon Bedrock for setting up API access.
Add Repositories and BOM
Spring AI artifacts are published in Spring Milestone and Snapshot repositories. Refer to the Repositories section to add these repositories to your build system.
To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the Dependency Management section to add the Spring AI BOM to your build system.
Auto-configuration
Add the spring-ai-bedrock-ai-spring-boot-starter
dependency to your project’s Maven pom.xml
file:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bedrock-ai-spring-boot-starter</artifactId>
</dependency>
or to your Gradle build.gradle
build file.
dependencies {
implementation 'org.springframework.ai:spring-ai-bedrock-ai-spring-boot-starter'
}
Refer to the Dependency Management section to add the Spring AI BOM to your build file. |
Enable Anthropic Chat
By default the Anthropic model is disabled.
To enable it set the spring.ai.bedrock.anthropic3.chat.enabled
property to true
.
Exporting environment variable is one way to set this configuration property:
export SPRING_AI_BEDROCK_ANTHROPIC3_CHAT_ENABLED=true
Chat Properties
The prefix spring.ai.bedrock.aws
is the property prefix to configure the connection to AWS Bedrock.
Property | Description | Default |
---|---|---|
spring.ai.bedrock.aws.region |
AWS region to use. |
us-east-1 |
spring.ai.bedrock.aws.timeout |
AWS timeout to use. |
5m |
spring.ai.bedrock.aws.access-key |
AWS access key. |
- |
spring.ai.bedrock.aws.secret-key |
AWS secret key. |
- |
The prefix spring.ai.bedrock.anthropic3.chat
is the property prefix that configures the chat model implementation for Claude.
Property | Description | Default |
---|---|---|
spring.ai.bedrock.anthropic3.chat.enabled |
Enable Bedrock Anthropic chat model. Disabled by default |
false |
spring.ai.bedrock.anthropic3.chat.model |
The model id to use. Supports the |
|
spring.ai.bedrock.anthropic3.chat.options.temperature |
Controls the randomness of the output. Values can range over [0.0,1.0] |
0.8 |
spring.ai.bedrock.anthropic3.chat.options.top-p |
The maximum cumulative probability of tokens to consider when sampling. |
AWS Bedrock default |
spring.ai.bedrock.anthropic3.chat.options.top-k |
Specify the number of token choices the generative uses to generate the next token. |
AWS Bedrock default |
spring.ai.bedrock.anthropic3.chat.options.stop-sequences |
Configure up to four sequences that the generative recognizes. After a stop sequence, the generative stops generating further tokens. The returned text doesn’t contain the stop sequence. |
10 |
spring.ai.bedrock.anthropic3.chat.options.anthropic-version |
The version of the generative to use. |
bedrock-2023-05-31 |
spring.ai.bedrock.anthropic3.chat.options.max-tokens |
Specify the maximum number of tokens to use in the generated response. Note that the models may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. We recommend a limit of 4,000 tokens for optimal performance. |
500 |
Look at the AnthropicChatModel for other model IDs.
Supported values are: anthropic.claude-instant-v1
, anthropic.claude-v2
and anthropic.claude-v2:1
.
Model ID values can also be found in the AWS Bedrock documentation for base model IDs.
All properties prefixed with spring.ai.bedrock.anthropic3.chat.options can be overridden at runtime by adding a request specific Runtime Options to the Prompt call.
|
Runtime Options
The Anthropic3ChatOptions.java provides model configurations, such as temperature, topK, topP, etc.
On start-up, the default options can be configured with the BedrockAnthropicChatModel(api, options)
constructor or the spring.ai.bedrock.anthropic3.chat.options.*
properties.
At run-time you can override the default options by adding new, request specific, options to the Prompt
call.
For example to override the default temperature for a specific request:
ChatResponse response = chatModel.call(
new Prompt(
"Generate the names of 5 famous pirates.",
Anthropic3ChatOptions.builder()
.withTemperature(0.4)
.build()
));
In addition to the model specific AnthropicChatOptions you can use a portable ChatOptions instance, created with the ChatOptionsBuilder#builder(). |
Multimodal
Multimodality refers to a model’s ability to simultaneously understand and process information from various sources, including text, images, audio, and other data formats. This paradigm represents a significant advancement in AI models.
Currently, Anthropic Claude 3 supports the base64
source type for images
, and the image/jpeg
, image/png
, image/gif
, and image/webp
media types.
Check the Vision guide for more information.
Spring AI’s Message
interface supports multimodal AI models by introducing the Media type.
This type contains data and information about media attachments in messages, using Spring’s org.springframework.util.MimeType
and a java.lang.Object
for the raw media data.
Below is a simple code example extracted from Anthropic3ChatModelIT.java, demonstrating the combination of user text with an image.
byte[] imageData = new ClassPathResource("/test.png").getContentAsByteArray();
var userMessage = new UserMessage("Explain what do you see o this picture?",
List.of(new Media(MimeTypeUtils.IMAGE_PNG, this.imageData)));
ChatResponse response = chatModel.call(new Prompt(List.of(this.userMessage)));
assertThat(response.getResult().getOutput().getContent()).contains("bananas", "apple", "basket");
It takes as an input the test.png
image:
along with the text message "Explain what do you see on this picture?", and generates a response something like:
The image shows a close-up view of a wire fruit basket containing several pieces of fruit. The basket appears to be made of thin metal wires formed into a round shape with an elevated handle. Inside the basket, there are a few yellow bananas and a couple of red apples or possibly tomatoes. The vibrant colors of the fruit contrast nicely against the metallic tones of the wire basket. The shallow depth of field in the photograph puts the focus squarely on the fruit in the foreground, while the basket handle extending upwards is slightly blurred, creating a pleasing bokeh effect in the background. The composition and lighting give the image a clean, minimalist aesthetic that highlights the natural beauty and freshness of the fruit displayed in this elegant wire basket.
Sample Controller
Create a new Spring Boot project and add the spring-ai-bedrock-ai-spring-boot-starter
to your pom (or gradle) dependencies.
Add a application.properties
file, under the src/main/resources
directory, to enable and configure the Anthropic chat model:
spring.ai.bedrock.aws.region=eu-central-1
spring.ai.bedrock.aws.timeout=1000ms
spring.ai.bedrock.aws.access-key=${AWS_ACCESS_KEY_ID}
spring.ai.bedrock.aws.secret-key=${AWS_SECRET_ACCESS_KEY}
spring.ai.bedrock.anthropic3.chat.enabled=true
spring.ai.bedrock.anthropic3.chat.options.temperature=0.8
spring.ai.bedrock.anthropic3.chat.options.top-k=15
replace the regions , access-key and secret-key with your AWS credentials.
|
This will create a BedrockAnthropicChatModel
implementation that you can inject into your class.
Here is an example of a simple @Controller
class that uses the chat model for text generations.
@RestController
public class ChatController {
private final BedrockAnthropic3ChatModel chatModel;
@Autowired
public ChatController(BedrockAnthropic3ChatModel chatModel) {
this.chatModel = chatModel;
}
@GetMapping("/ai/generate")
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
return Map.of("generation", this.chatModel.call(message));
}
@GetMapping("/ai/generateStream")
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
Prompt prompt = new Prompt(new UserMessage(message));
return this.chatModel.stream(prompt);
}
}
Manual Configuration
The BedrockAnthropic3ChatModel implements the ChatModel
and StreamingChatModel
and uses the Low-level Anthropic3ChatBedrockApi Client to connect to the Bedrock Anthropic service.
Add the spring-ai-bedrock
dependency to your project’s Maven pom.xml
file:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bedrock</artifactId>
</dependency>
or to your Gradle build.gradle
build file.
dependencies {
implementation 'org.springframework.ai:spring-ai-bedrock'
}
Refer to the Dependency Management section to add the Spring AI BOM to your build file. |
Next, create an BedrockAnthropic3ChatModel and use it for text generations:
Anthropic3ChatBedrockApi anthropicApi = new Anthropic3ChatBedrockApi(
AnthropicChatBedrockApi.AnthropicModel.CLAUDE_V3_SONNET.id(),
EnvironmentVariableCredentialsProvider.create(),
Region.US_EAST_1.id(),
new ObjectMapper(),
Duration.ofMillis(1000L));
BedrockAnthropic3ChatModel chatModel = new BedrockAnthropic3ChatModel(this.anthropicApi,
AnthropicChatOptions.builder()
.withTemperature(0.6)
.withTopK(10)
.withTopP(0.8)
.withMaxTokensToSample(100)
.withAnthropicVersion(AnthropicChatBedrockApi.DEFAULT_ANTHROPIC_VERSION)
.build());
ChatResponse response = this.chatModel.call(
new Prompt("Generate the names of 5 famous pirates."));
// Or with streaming responses
Flux<ChatResponse> response = this.chatModel.stream(
new Prompt("Generate the names of 5 famous pirates."));
Low-level Anthropic3ChatBedrockApi Client
The Anthropic3ChatBedrockApi provides is lightweight Java client on top of AWS Bedrock Anthropic Claude models.
Client supports the anthropic.claude-3-opus-20240229-v1:0
,anthropic.claude-3-sonnet-20240229-v1:0
,anthropic.claude-3-haiku-20240307-v1:0
and the legacy anthropic.claude-v2
, anthropic.claude-v2:1
and anthropic.claude-instant-v1
models for both synchronous (e.g. chatCompletion()
) and streaming (e.g. chatCompletionStream()
) responses.
Here is a simple snippet how to use the api programmatically:
Anthropic3ChatBedrockApi anthropicChatApi = new Anthropic3ChatBedrockApi(
AnthropicModel.CLAUDE_V2.id(), Region.US_EAST_1.id(), Duration.ofMillis(1000L));
AnthropicChatRequest request = AnthropicChatRequest
.builder(String.format(Anthropic3ChatBedrockApi.PROMPT_TEMPLATE, "Name 3 famous pirates"))
.withTemperature(0.8)
.withMaxTokensToSample(300)
.withTopK(10)
.build();
// Sync request
AnthropicChatResponse response = this.anthropicChatApi.chatCompletion(this.request);
// Streaming request
Flux<AnthropicChatResponse> responseStream = this.anthropicChatApi.chatCompletionStream(this.request);
List<AnthropicChatResponse> responses = this.responseStream.collectList().block();
Follow the Anthropic3ChatBedrockApi.java's JavaDoc for further information.