Class HuggingfaceChatModel

java.lang.Object

org.springframework.ai.huggingface.HuggingfaceChatModel

All Implemented Interfaces:

ChatModel, StreamingChatModel, Model<Prompt, ChatResponse>, StreamingModel<Prompt, ChatResponse>

public class HuggingfaceChatModel
extends Object
implements ChatModel

An implementation of ChatModel that interfaces with HuggingFace Inference Endpoints for text generation.

Author:

Mark Pollack, Jihoon Kim

Constructor Summary

Constructors

Constructor	Description
HuggingfaceChatModel(String apiToken, String basePath)	Constructs a new HuggingfaceChatModel with the specified API token and base path.

Method Summary

Modifier and Type	Method	Description
ChatResponse	call(Prompt prompt)	Generate text based on the provided prompt.
int	getMaxNewTokens()	Gets the maximum number of new tokens to be generated.
void	setMaxNewTokens(int maxNewTokens)	Sets the maximum number of new tokens to be generated.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ChatModel

call, call, getDefaultOptions, stream

Methods inherited from interface StreamingChatModel

stream, stream

Constructor Details

HuggingfaceChatModel

public HuggingfaceChatModel(String apiToken,
 String basePath)

Constructs a new HuggingfaceChatModel with the specified API token and base path.

Parameters:

apiToken - The API token for HuggingFace.

basePath - The base path for API requests.

Method Details

call

public ChatResponse call(Prompt prompt)

Generate text based on the provided prompt.

Specified by:

call in interface ChatModel

Specified by:

call in interface Model<Prompt, ChatResponse>

Parameters:

prompt - The input prompt based on which text is to be generated.

Returns:

ChatResponse containing the generated text and other related details.

getMaxNewTokens

public int getMaxNewTokens()

Gets the maximum number of new tokens to be generated.

Returns:

The maximum number of new tokens.

setMaxNewTokens

public void setMaxNewTokens(int maxNewTokens)

Sets the maximum number of new tokens to be generated.

Parameters:

maxNewTokens - The maximum number of new tokens.