This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.1.0!

Audio Models

Spring AI has two audio model types: transcription turns speech into text, and text-to-speech turns text into speech. Both can stream.

Model type Interface and main methods Implementations

Transcription

TranscriptionModel: transcribe(Resource) returns a String; streamTranscribe(Resource) returns a Flux<String>.

OpenAI, Google GenAI, Qxotic jinfer (Community)

Text to speech

TextToSpeechModel: call(String) returns the audio as a byte[]; stream(String) returns a Flux<byte[]>.

OpenAI, ElevenLabs, Google GenAI, Qxotic jinfer (Community)

Each model also has a method that takes a prompt with options and returns a response with metadata: call(AudioTranscriptionPrompt) and call(TextToSpeechPrompt). The pages of the two model types describe these, and how to write code that works with any provider.

A chat model that accepts audio input can also answer questions about a recording, see Multimodality.