Package org.springframework.ai.bedrock.converse

Class BedrockProxyChatModel

java.lang.Object

org.springframework.ai.chat.model.AbstractToolCallSupport

org.springframework.ai.bedrock.converse.BedrockProxyChatModel

All Implemented Interfaces:

ChatModel, StreamingChatModel, Model<Prompt,ChatResponse>, StreamingModel<Prompt,ChatResponse>

public class BedrockProxyChatModel
extends AbstractToolCallSupport
implements ChatModel

A ChatModel implementation that uses the Amazon Bedrock Converse API to interact with the Supported models.

The Converse API doesn't support any embedding models (such as Titan Embeddings G1 - Text) or image generation models (such as Stability AI).

https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html

https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html

https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html

https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html

https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html

Since:

1.0.0

Author:

Christian Tzolov, Wei Jiang

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final class	BedrockProxyChatModel.Builder

Field Summary

Fields

Modifier and Type	Field	Description
static final reactor.core.publisher.Sinks.EmitFailureHandler	DEFAULT_EMIT_FAILURE_HANDLER

Fields inherited from class org.springframework.ai.chat.model.AbstractToolCallSupport

functionCallbackContext, functionCallbackRegister, IS_RUNTIME_CALL

Constructor Summary

Constructors

Constructor

Description

BedrockProxyChatModel(software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient bedrockRuntimeClient, software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeAsyncClient bedrockRuntimeAsyncClient, FunctionCallingOptions defaultOptions, FunctionCallbackContext functionCallbackContext, List<FunctionCallback> toolFunctionCallbacks, io.micrometer.observation.ObservationRegistry observationRegistry)

Method Summary

Modifier and Type	Method	Description
static BedrockProxyChatModel.Builder	builder()
ChatResponse	call(Prompt prompt)	Invoke the model and return the response.
reactor.core.publisher.Flux<software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamOutput>	converseStream(software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamRequest converseStreamRequest)	Invoke the model and return the response stream.
software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamRequest	createStreamRequest(Prompt prompt)
ChatOptions	getDefaultOptions()
void	setObservationConvention(ChatModelObservationConvention observationConvention)	Use the provided convention for reporting observation data
reactor.core.publisher.Flux<ChatResponse>	stream(Prompt prompt)	Invoke the model and return the response stream.

Methods inherited from class org.springframework.ai.chat.model.AbstractToolCallSupport

buildToolCallConversation, executeFunctions, getFunctionCallbackRegister, handleToolCalls, isProxyToolCalls, isToolCall, isToolCall, resolveFunctionCallbacks, runtimeFunctionCallbackConfigurations

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.springframework.ai.chat.model.ChatModel

call, call

Methods inherited from interface org.springframework.ai.chat.model.StreamingChatModel

stream, stream

Field Details

DEFAULT_EMIT_FAILURE_HANDLER

public static final reactor.core.publisher.Sinks.EmitFailureHandler DEFAULT_EMIT_FAILURE_HANDLER

Constructor Details

BedrockProxyChatModel

public BedrockProxyChatModel(software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient bedrockRuntimeClient,
 software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeAsyncClient bedrockRuntimeAsyncClient,
 FunctionCallingOptions defaultOptions,
 FunctionCallbackContext functionCallbackContext,
 List<FunctionCallback> toolFunctionCallbacks,
 io.micrometer.observation.ObservationRegistry observationRegistry)

Method Details

call

public ChatResponse call(Prompt prompt)

Invoke the model and return the response. https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeClient.html#converse

Specified by:

call in interface ChatModel

Specified by:

call in interface Model<Prompt,ChatResponse>

Parameters:

prompt - the request object to be sent to the AI model

Returns:

The model invocation response.

getDefaultOptions

public ChatOptions getDefaultOptions()

Specified by:

getDefaultOptions in interface ChatModel

createStreamRequest

public software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamRequest createStreamRequest(Prompt prompt)

stream

public reactor.core.publisher.Flux<ChatResponse> stream(Prompt prompt)

Invoke the model and return the response stream. https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeAsyncClient.html#converseStream

Specified by:

stream in interface ChatModel

Specified by:

stream in interface StreamingChatModel

Specified by:

stream in interface StreamingModel<Prompt,ChatResponse>

Parameters:

prompt - the request object to be sent to the AI model

Returns:

The model invocation response stream.

converseStream

public reactor.core.publisher.Flux<software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamOutput> converseStream(software.amazon.awssdk.services.bedrockruntime.model.ConverseStreamRequest converseStreamRequest)

Invoke the model and return the response stream. https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeAsyncClient.html#converseStream

Parameters:

converseStreamRequest - Model invocation request.

Returns:

The model invocation response stream.

setObservationConvention

public void setObservationConvention(ChatModelObservationConvention observationConvention)

Use the provided convention for reporting observation data

Parameters:

observationConvention - The provided convention

builder

public static BedrockProxyChatModel.Builder builder()