Package org.springframework.ai.bedrock.api

Class AbstractBedrockApi<I,O,SO>

java.lang.Object
org.springframework.ai.bedrock.api.AbstractBedrockApi<I,O,SO>
Type Parameters:
I - The input request type.
O - The output response type.
SO - The streaming response type. For some models this type can be the same as the output response type.
Direct Known Subclasses:
CohereEmbeddingBedrockApi, TitanEmbeddingBedrockApi
public abstract class AbstractBedrockApi<I,O,SO> extends Object
Abstract class for the Bedrock API. It provides the basic functionality to invoke the chat completion model and receive the response for streaming and non-streaming requests.

https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html

https://us-east-1.console.aws.amazon.com/bedrock/home?region=us-east-1#/modelaccess

Since:
0.8.0
Author:
Christian Tzolov, Wei Jiang
See Also:

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final record 
    AbstractBedrockApi.AmazonBedrockInvocationMetrics
    Encapsulates the metrics about the model invocation.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final reactor.core.publisher.Sinks.EmitFailureHandler
    DEFAULT_EMIT_FAILURE_HANDLER
    Default emit failure handler.

  • Constructor Summary

    Constructors
    Constructor
    Description
    AbstractBedrockApi(String modelId, String region)
    Create a new AbstractBedrockApi instance using default credentials provider and object mapper.
    AbstractBedrockApi(String modelId, String region, Duration timeout)
    Create a new AbstractBedrockApi instance using default credentials provider and object mapper.
    AbstractBedrockApi(String modelId, software.amazon.awssdk.auth.credentials.AwsCredentialsProvider credentialsProvider, String region, com.fasterxml.jackson.databind.ObjectMapper objectMapper)
    Create a new AbstractBedrockApi instance using the provided credentials provider, region and object mapper.
    AbstractBedrockApi(String modelId, software.amazon.awssdk.auth.credentials.AwsCredentialsProvider credentialsProvider, String region, com.fasterxml.jackson.databind.ObjectMapper objectMapper, Duration timeout)
    Create a new AbstractBedrockApi instance using the provided credentials provider, region and object mapper.
    AbstractBedrockApi(String modelId, software.amazon.awssdk.auth.credentials.AwsCredentialsProvider credentialsProvider, software.amazon.awssdk.regions.Region region, com.fasterxml.jackson.databind.ObjectMapper objectMapper, Duration timeout)
    Create a new AbstractBedrockApi instance using the provided credentials provider, region and object mapper.

  • Method Summary

    Modifier and Type
    Method
    Description
    protected O
    chatCompletion(I request)
    Chat completion invocation.
    protected reactor.core.publisher.Flux<SO>
    chatCompletionStream(I request)
    Chat completion invocation with streaming response.
    protected O
    embedding(I request)
    Compute the embedding for the given text.
    String
    getModelId()
    Get the model id.
    software.amazon.awssdk.regions.Region
    getRegion()
    Get the AWS region.
    protected O
    internalInvocation(I request, Class<O> clazz)
    Internal method to invoke the model and return the response.
    protected reactor.core.publisher.Flux<SO>
    internalInvocationStream(I request, Class<SO> clazz)
    Internal method to invoke the model and return the response stream.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • DEFAULT_EMIT_FAILURE_HANDLER

      public static final reactor.core.publisher.Sinks.EmitFailureHandler DEFAULT_EMIT_FAILURE_HANDLER
      Default emit failure handler.

  • Constructor Details

    • AbstractBedrockApi

      public AbstractBedrockApi(String modelId, String region)
      Create a new AbstractBedrockApi instance using default credentials provider and object mapper.
      Parameters:
      modelId - The model id to use.
      region - The AWS region to use.

    • AbstractBedrockApi

      public AbstractBedrockApi(String modelId, String region, Duration timeout)
      Create a new AbstractBedrockApi instance using default credentials provider and object mapper.
      Parameters:
      modelId - The model id to use.
      region - The AWS region to use.
      timeout - The timeout to use.

    • AbstractBedrockApi

      public AbstractBedrockApi(String modelId, software.amazon.awssdk.auth.credentials.AwsCredentialsProvider credentialsProvider, String region, com.fasterxml.jackson.databind.ObjectMapper objectMapper)
      Create a new AbstractBedrockApi instance using the provided credentials provider, region and object mapper.
      Parameters:
      modelId - The model id to use.
      credentialsProvider - The credentials provider to connect to AWS.
      region - The AWS region to use.
      objectMapper - The object mapper to use for JSON serialization and deserialization.

    • AbstractBedrockApi

      public AbstractBedrockApi(String modelId, software.amazon.awssdk.auth.credentials.AwsCredentialsProvider credentialsProvider, String region, com.fasterxml.jackson.databind.ObjectMapper objectMapper, Duration timeout)
      Create a new AbstractBedrockApi instance using the provided credentials provider, region and object mapper.
      Parameters:
      modelId - The model id to use.
      credentialsProvider - The credentials provider to connect to AWS.
      region - The AWS region to use.
      objectMapper - The object mapper to use for JSON serialization and deserialization.
      timeout - Configure the amount of time to allow the client to complete the execution of an API call. This timeout covers the entire client execution except for marshalling. This includes request handler execution, all HTTP requests including retries, unmarshalling, etc. This value should always be positive, if present.

    • AbstractBedrockApi

      public AbstractBedrockApi(String modelId, software.amazon.awssdk.auth.credentials.AwsCredentialsProvider credentialsProvider, software.amazon.awssdk.regions.Region region, com.fasterxml.jackson.databind.ObjectMapper objectMapper, Duration timeout)
      Create a new AbstractBedrockApi instance using the provided credentials provider, region and object mapper.
      Parameters:
      modelId - The model id to use.
      credentialsProvider - The credentials provider to connect to AWS.
      region - The AWS region to use.
      objectMapper - The object mapper to use for JSON serialization and deserialization.
      timeout - Configure the amount of time to allow the client to complete the execution of an API call. This timeout covers the entire client execution except for marshalling. This includes request handler execution, all HTTP requests including retries, unmarshalling, etc. This value should always be positive, if present.

  • Method Details

    • getModelId

      public String getModelId()
      Get the model id.
      Returns:
      The model id.

    • getRegion

      public software.amazon.awssdk.regions.Region getRegion()
      Get the AWS region.
      Returns:
      The AWS region.

    • embedding

      protected O embedding(I request)
      Compute the embedding for the given text.
      Parameters:
      request - The embedding request.
      Returns:
      Returns the embedding response.

    • chatCompletion

      protected O chatCompletion(I request)
      Chat completion invocation.
      Parameters:
      request - The chat completion request.
      Returns:
      The chat completion response.

    • chatCompletionStream

      protected reactor.core.publisher.Flux<SO> chatCompletionStream(I request)
      Chat completion invocation with streaming response.
      Parameters:
      request - The chat completion request.
      Returns:
      The chat completion response stream.

    • internalInvocation

      protected O internalInvocation(I request, Class<O> clazz)
      Internal method to invoke the model and return the response. https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/bedrockruntime/BedrockRuntimeClient.html#invokeModel
      Parameters:
      request - Model invocation request.
      clazz - The response class type
      Returns:
      The model invocation response.

    • internalInvocationStream

      protected reactor.core.publisher.Flux<SO> internalInvocationStream(I request, Class<SO> clazz)
      Internal method to invoke the model and return the response stream.
      Parameters:
      request - Model invocation request.
      clazz - Response class type.
      Returns:
      The model invocation response stream.