|
This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.1.0! |
Vector Store API
A vector database is a specialized type of database that plays an essential role in AI applications.
In vector databases, queries differ from traditional relational databases. Instead of exact matches, they perform similarity searches. When given a vector as a query, a vector database returns vectors that are “similar” to the query vector. Further details on how this similarity is calculated at a high-level is provided in a Vector Similarity.
Vector databases are used to integrate your data with AI models. The first step in their usage is to load your data into a vector database. Then, when a user query is to be sent to the AI model, a set of similar documents is first retrieved. These documents then serve as the context for the user’s question and are sent to the AI model, along with the user’s query. This technique is known as Retrieval Augmented Generation (RAG).
The following sections describe the Spring AI interface for using multiple vector database implementations and some high-level sample usage.
The last section is intended to demystify the underlying approach of similarity searching in vector databases.
API Overview
This section serves as a guide to the VectorStore interface and its associated classes within the Spring AI framework.
Spring AI offers an abstracted API for interacting with vector databases through the VectorStore interface and its read-only counterpart, the VectorStoreRetriever interface.
VectorStoreRetriever Interface
Spring AI provides a read-only interface called VectorStoreRetriever that exposes only the document retrieval functionality:
@FunctionalInterface
public interface VectorStoreRetriever {
List<Document> similaritySearch(SearchRequest request);
default List<Document> similaritySearch(String query) {
return this.similaritySearch(SearchRequest.builder().query(query).build());
}
default List<Document> similaritySearch(float[] queryEmbedding, SearchRequest request) { ... }
}
This functional interface is designed for use cases where you only need to retrieve documents from a vector store without performing any mutation operations. It follows the principle of least privilege by exposing only the necessary functionality for document retrieval.
VectorStore Interface
The VectorStore interface extends VectorStoreRetriever and adds mutation capabilities:
public interface VectorStore extends DocumentWriter, VectorStoreRetriever {
default String getName() {
return this.getClass().getSimpleName();
}
void add(List<Document> documents);
default void upsert(List<EmbeddedDocument> entries) { ... }
void delete(List<String> idList);
void delete(Filter.Expression filterExpression);
default void delete(String filterExpression) { ... }
default <T> Optional<T> getNativeClient() {
return Optional.empty();
}
}
The VectorStore interface combines both read and write operations, allowing you to add, delete, and search for documents in a vector database.
While add computes the embeddings for you, upsert writes ones you supply yourself; see Writing User-Supplied Embeddings.
SearchRequest Builder
public class SearchRequest {
public static final double SIMILARITY_THRESHOLD_ACCEPT_ALL = 0.0;
public static final int DEFAULT_TOP_K = 4;
private String query = "";
private int topK = DEFAULT_TOP_K;
private double similarityThreshold = SIMILARITY_THRESHOLD_ACCEPT_ALL;
@Nullable
private Filter.Expression filterExpression;
public static Builder from(SearchRequest originalSearchRequest) {
return builder().query(originalSearchRequest.getQuery())
.topK(originalSearchRequest.getTopK())
.similarityThreshold(originalSearchRequest.getSimilarityThreshold())
.filterExpression(originalSearchRequest.getFilterExpression());
}
public static class Builder {
private final SearchRequest searchRequest = new SearchRequest();
public Builder query(String query) {
Assert.notNull(query, "Query can not be null.");
this.searchRequest.query = query;
return this;
}
public Builder topK(int topK) {
Assert.isTrue(topK >= 0, "TopK should be positive.");
this.searchRequest.topK = topK;
return this;
}
public Builder similarityThreshold(double threshold) {
Assert.isTrue(threshold >= 0 && threshold <= 1, "Similarity threshold must be in [0,1] range.");
this.searchRequest.similarityThreshold = threshold;
return this;
}
public Builder similarityThresholdAll() {
this.searchRequest.similarityThreshold = 0.0;
return this;
}
public Builder filterExpression(@Nullable Filter.Expression expression) {
this.searchRequest.filterExpression = expression;
return this;
}
public Builder filterExpression(@Nullable String textExpression) {
this.searchRequest.filterExpression = (textExpression != null)
? new FilterExpressionTextParser().parse(textExpression) : null;
return this;
}
public SearchRequest build() {
return this.searchRequest;
}
}
public String getQuery() {...}
public int getTopK() {...}
public double getSimilarityThreshold() {...}
public Filter.Expression getFilterExpression() {...}
}
To insert data into the vector database, encapsulate it within a Document object.
The Document class encapsulates content from a data source, such as a PDF or Word document, and includes text represented as a string.
It also contains metadata in the form of key-value pairs, including details such as the filename.
Upon insertion into the vector database, the text content is transformed into a numerical array, or a float[], known as vector embeddings, using an embedding model.
Embedding models, such as Word2Vec, GLoVE, and BERT, or OpenAI’s text-embedding-ada-002, are used to convert words, sentences, or paragraphs into these vector embeddings.
The vector database’s role is to store and facilitate similarity searches for these embeddings.
It does not generate the embeddings itself.
For creating vector embeddings, the EmbeddingModel should be utilized.
The similaritySearch methods in the interface allow for retrieving documents similar to a given query string.
These methods can be fine-tuned by using the following parameters:
-
k: An integer that specifies the maximum number of similar documents to return. This is often referred to as a 'top K' search, or 'K nearest neighbors' (KNN). -
threshold: A double value ranging from 0 to 1, where values closer to 1 indicate higher similarity. By default, if you set a threshold of 0.75, for instance, only documents with a similarity above this value are returned. -
Filter.Expression: A class used for passing a fluent DSL (Domain-Specific Language) expression that functions similarly to a 'where' clause in SQL, but it applies exclusively to the metadata key-value pairs of aDocument. -
filterExpression: An external DSL based on ANTLR4 that accepts filter expressions as strings. For example, with metadata keys like country, year, andisActive, you could use an expression such as:country == 'UK' && year >= 2020 && isActive == true.
Find more information on the Filter.Expression in the Metadata Filters section.
Example Usage
To compute the embeddings for a vector database, you need to pick an embedding model that matches the higher-level AI model being used.
For example, with OpenAI’s ChatGPT, we use the OpenAiEmbeddingModel and a model named text-embedding-ada-002.
The Spring Boot starter’s auto-configuration for OpenAI makes an implementation of EmbeddingModel available in the Spring application context for dependency injection.
Writing to a Vector Store
The general usage of loading data into a vector store is something you would do in a batch-like job, by first loading data into Spring AI’s Document class and then calling the add method on the VectorStore interface.
Given a String reference to a source file that represents a JSON file with data we want to load into the vector database, we use Spring AI’s JsonReader to load specific fields in the JSON, which splits them up into small pieces and then passes those small pieces to the vector store implementation.
The VectorStore implementation computes the embeddings and stores the JSON and the embedding in the vector database:
@Autowired
VectorStore vectorStore;
void load(String sourceFile) {
JsonReader jsonReader = new JsonReader(new FileSystemResource(sourceFile),
"price", "name", "shortDescription", "description", "tags");
List<Document> documents = jsonReader.get();
this.vectorStore.add(documents);
}
add computes the embeddings for you.
If you already have the vectors, write them as they are with upsert instead; see Writing User-Supplied Embeddings.
Writing User-Supplied Embeddings
add computes an embedding for every document inside the store, using the store’s embedding model.
Sometimes you already have the vectors and want to write them without embedding anything again: a provider’s batch API computed them overnight at half price, another team owns the embedding pipeline, or you are migrating from a system that exports text and vectors together.
For those cases the VectorStore interface offers upsert, which takes the document and its vector paired together in an EmbeddedDocument and never embeds anything itself:
public record EmbeddedDocument(Document document, float[] embedding) {
}
As the name says, upsert replaces by id: writing the same id again overwrites the existing row rather than adding a duplicate.
That makes a re-run with stable ids safe, which is useful for an ingestion job that may be retried after a failure.
@Autowired
VectorStore vectorStore;
// a vector you computed elsewhere
float[] embedding = ...;
Document document = new Document("8f14e45f-ceea-467a-9a3e-5b1c2d6f7a90", "some text",
Map.of("source", "manual"));
this.vectorStore.upsert(List.of(new EmbeddedDocument(document, embedding)));
Some stores constrain the document id.
Qdrant requires a UUID, and PgVectorStore stores the id in a UUID column unless you configure idType, so an arbitrary string fails on those.
Deriving the UUID from a key of your own, for example UUID.nameUUIDFromBytes(key.getBytes(UTF_8)), keeps the id stable across runs, which is what makes a re-run replace the row rather than add a second one.
|
To write plain text through upsert, embed it first and hand over the result:
@Autowired
EmbeddingModel embeddingModel;
float[] vector = this.embeddingModel.embed(document);
this.vectorStore.upsert(List.of(new EmbeddedDocument(document, vector)));
upsert is opt-in per store: the default implementation throws UnsupportedOperationException, so a store supports it only once it has been implemented.
It is available on pgvector, Redis, Elasticsearch, Qdrant, OpenSearch, Chroma, MariaDB, MongoDB Atlas, Typesense, Neo4j, and Couchbase, with more stores to follow.
A custom store that extends AbstractObservationVectorStore opts in by implementing doUpsert and overriding supportsUpsert() to return true; for search with a query embedding, the pair is doSimilaritySearch(float[], SearchRequest) and supportsQueryEmbeddingSearch().
EmbeddedDocument rejects a vector that is empty or holds NaN or an infinity, which is worth knowing when the vectors come from a pipeline you do not control: a NaN would otherwise corrupt distance maths rather than fail.
Some stores normalize vectors for cosine similarity, Redis and Qdrant among them, so what a store keeps may differ in length from what you wrote while pointing the same way.
Searching is unaffected, but do not rely on reading the exact original vector back from the store’s native client.
When a store knows its index dimension, it checks every vector in the batch against it before writing anything, so a wrong-sized vector fails fast and cannot partially write.
The store works out that dimension from a dimensions value you configured, or otherwise reads it from the existing index, table, or collection.
It asks the embedding model for a width only when it has to create the index itself, and never to check vectors against an index that already exists, because the model’s width says nothing about the width of an existing index.
So an application that only upserts into an existing index, or that configures dimensions, never contacts the embedding model and does not need to be able to reach it.
Where the dimension cannot be determined, for example because the index cannot be read right now, the check is left to the database, which rejects the vector with its own error.
The store tries to read the dimension again on the next call.
A pgvector column declared as a plain vector, without a size, accepts any width, so nothing is checked there either.
Redis cannot leave the check to the database: it stores a wrong-sized vector without complaint and never finds it again, so RedisVectorStore refuses the write instead.
Once the dimension is known, the store remembers it, so create a new store instance after you recreate an index with a different width.
Because the vectors come from outside the store, lining them up with the index is yours to get right. Three things have to agree, and only the first is checked for you:
- Width
-
Your embedding model decides the width, and the index has to be created with that same width, for example
dimensions(1024)on the store builder or a pre-created index whose mapping says 1024. A mismatch fails on the first write. - Vector space
-
Nothing records which model produced a row, so vectors written by one model and searched with a model from a different embedding space come back plausible but meaningless, with no error anywhere. Writing the model name and version into metadata costs nothing and tells a later reader what a row actually holds.
- The query side
-
similaritySearchwith query text embeds the query with the store’s ownEmbeddingModel, so that model has to produce vectors in the same space and of the same width as the ones you upserted. To keep the store’s model out of it, embed the query yourself with the same model and search with the vector; see Searching with a User-Supplied Query Embedding.
Indexing Non-Text or Oversized Content
With text, the store keeps the document and its vector together in one row: search matches on the vector and hands back the text. A multimodal model breaks that assumption. It gives you a searchable vector for an image, a video segment, or an audio clip, but the content itself lives in S3, a CDN, or a database, and there is no text to put in the row. The same applies to text that is simply too large for the row, such as a full article whose chunks were indexed.
The row then keeps everything except the content: id, metadata, vector, and a pointer to where the content lives.
The pointer goes in metadata under DocumentMetadata.CONTENT_REF, and the store never interprets it: it is an S3 URI, a CDN URL, or a database key that only your application resolves.
Such a row is written with upsert, because you supply the vector from the multimodal model yourself:
byte[] frame = ...; // the image/video/audio bytes
float[] embedding = multimodalModel.embed(frame); // your multimodal model
String key = "video-42/frame-4711";
Document reference = Document.builder()
// Stable and UUID-shaped, so a re-run replaces this row
.id(UUID.nameUUIDFromBytes(key.getBytes(UTF_8)).toString())
.text("") // empty: the content lives elsewhere
.metadata(DocumentMetadata.CONTENT_REF.value(), "s3://media/video-42.mp4#t=4711")
.metadata("camera", "north-gate") // still filterable like any row
.build();
this.vectorStore.upsert(List.of(new EmbeddedDocument(reference, embedding)));
The empty text is what makes the document valid; it is not content.
A document carrying CONTENT_REF is a reference row even though isText() reports true.
upsert requires text for this reason: no store can persist a row that carries none, so a media document is rejected rather than half-written.
|
The pointer is an ordinary metadata entry.
Nothing in the framework reads or validates it, so keeping it non-empty is up to you.
A row that still has real text of its own can be written with add, which is how a summary or a chunk keeps a pointer back to the full article:
Document chunk = Document.builder()
.text(chunkText) // real text, so add can embed it
.metadata(DocumentMetadata.CONTENT_REF.value(), "articles/1042") // where the full article lives
.build();
this.vectorStore.add(List.of(chunk));
Only a row with empty text has to go through upsert, because add embeds the text and there would be nothing to embed.
Search is unchanged.
similaritySearch matches on vectors and returns Document s as always, and for these rows the caller follows the pointer:
List<Document> results = this.vectorStore.similaritySearch(SearchRequest.builder()
.query("a truck at the north gate after dark") // embedded by the same multimodal model
.filterExpression("camera == 'north-gate'")
.build());
for (Document doc : results) {
String ref = (String) doc.getMetadata().get(DocumentMetadata.CONTENT_REF.value());
byte[] frame = mediaStorage.fetch(ref); // S3, CDN, wherever the pointer leads
}
Two models are in play here and they have to produce vectors in the same space and of the same width: the one you embed the content with, and the store’s own EmbeddingModel, which embeds the query inside similaritySearch.
Spring AI’s VertexAiMultimodalEmbeddingModel puts text and images in one 1408-dimension space, but it implements DocumentEmbeddingModel rather than EmbeddingModel, so using it on both sides means wrapping it for the store.
Wire an ordinary text model instead and your queries land somewhere else entirely: an error if the widths differ, and quietly meaningless results if they happen to match.
|
Redis returns only the metadata fields declared when the store is built, so content_ref has to be declared for the pointer to survive a read: .metadataFields(MetadataField.tag(DocumentMetadata.CONTENT_REF.value())).
The same goes for any field you filter on, such as camera above.
Adding a field to a store that already has an index means recreating the index.
|
Reading from a Vector Store
Later, when a user question is passed into the AI model, a similarity search is done to retrieve similar documents, which are then "stuffed" into the prompt as context for the user’s question.
For read-only operations, you can use either the VectorStore interface or the more focused VectorStoreRetriever interface:
@Autowired
VectorStoreRetriever retriever; // Could also use VectorStore here
String question = "<question from user>";
List<Document> similarDocuments = retriever.similaritySearch(question);
// Or with more specific search parameters
SearchRequest request = SearchRequest.builder()
.query(question)
.topK(5) // Return top 5 results
.similarityThreshold(0.7) // Only return results with similarity score >= 0.7
.build();
List<Document> filteredDocuments = retriever.similaritySearch(request);
Additional options can be passed into the similaritySearch method to define how many documents to retrieve and a threshold of the similarity search.
Searching with a User-Supplied Query Embedding
Searching with text embeds the query with the store’s own EmbeddingModel.
If your application computes its embeddings itself, for example because the vectors were written with upsert, it can embed the query the same way and search with the vector:
float[] queryEmbedding = myEmbeddingPipeline.embed(question);
SearchRequest request = SearchRequest.builder()
.topK(5)
.similarityThreshold(0.7)
.filterExpression("tenant == 'acme'")
.build();
List<Document> documents = retriever.similaritySearch(queryEmbedding, request);
The request supplies everything except the query: topK, the similarity threshold, and the filter expression.
Its query text, if set, is ignored.
The store’s EmbeddingModel is never used, so an application that writes with upsert and searches this way does not need to be able to reach it.
The query embedding has to come from a compatible encoder in the same embedding space as the vectors it is compared against.
That is usually the same model, but some providers expect different settings for documents and for queries, such as a separate input type for each, so use the setting that is meant for queries.
A query vector from a different embedding space is not rejected when its width happens to match; the search simply returns results that look plausible and mean nothing.
What is checked for you: the vector must not be empty or contain NaN or infinity, and its width must match the store’s dimension when the store knows it.
Searching with a query embedding is opt-in per store, like upsert: the default implementation throws UnsupportedOperationException.
It is available on pgvector, Redis, Elasticsearch, Qdrant, OpenSearch, Chroma, MariaDB, MongoDB Atlas, Typesense, Neo4j, and Couchbase.
Separation of Read and Write Operations
Using the separate interfaces allows you to clearly define which components need write access and which only need read access:
// Write operations in a service that needs full access
@Service
class DocumentIndexer {
private final VectorStore vectorStore;
DocumentIndexer(VectorStore vectorStore) {
this.vectorStore = vectorStore;
}
public void indexDocuments(List<Document> documents) {
vectorStore.add(documents);
}
}
// Read-only operations in a service that only needs retrieval
@Service
class DocumentRetriever {
private final VectorStoreRetriever retriever;
DocumentRetriever(VectorStoreRetriever retriever) {
this.retriever = retriever;
}
public List<Document> findSimilar(String query) {
return retriever.similaritySearch(query);
}
}
This separation of concerns helps create more maintainable and secure applications by limiting access to mutation operations only to components that truly need them.
Retrieval Operations with VectorStoreRetriever
The VectorStoreRetriever interface provides a read-only view of a vector store, exposing only the similarity search functionality.
This follows the principle of least privilege and is particularly useful in RAG (Retrieval-Augmented Generation) applications where you only need to retrieve documents without modifying the underlying data.
Benefits of Using VectorStoreRetriever
-
Separation of Concerns: Clearly separates read operations from write operations.
-
Interface Segregation: Clients that only need retrieval functionality aren’t exposed to mutation methods.
-
Functional Interface: Can be implemented with lambda expressions or method references for simple use cases.
-
Reduced Dependencies: Components that only need to perform searches don’t need to depend on the full
VectorStoreinterface.
Example Usage
You can use VectorStoreRetriever directly when you only need to perform similarity searches:
@Service
public class DocumentRetrievalService {
private final VectorStoreRetriever retriever;
public DocumentRetrievalService(VectorStoreRetriever retriever) {
this.retriever = retriever;
}
public List<Document> findSimilarDocuments(String query) {
return retriever.similaritySearch(query);
}
public List<Document> findSimilarDocumentsWithFilters(String query, String country) {
SearchRequest request = SearchRequest.builder()
.query(query)
.topK(5)
.filterExpression("country == '" + country + "'")
.build();
return retriever.similaritySearch(request);
}
}
In this example, the service only depends on the VectorStoreRetriever interface, making it clear that it only performs retrieval operations and doesn’t modify the vector store.
Integration with RAG Applications
The VectorStoreRetriever interface is particularly useful in RAG applications, where you need to retrieve relevant documents to provide context for an AI model:
@Service
public class RagService {
private final VectorStoreRetriever retriever;
private final ChatModel chatModel;
public RagService(VectorStoreRetriever retriever, ChatModel chatModel) {
this.retriever = retriever;
this.chatModel = chatModel;
}
public String generateResponse(String userQuery) {
// Retrieve relevant documents
List<Document> relevantDocs = retriever.similaritySearch(userQuery);
// Extract content from documents to use as context
String context = relevantDocs.stream()
.map(Document::getText)
.collect(Collectors.joining("\n\n"));
// Generate response using the retrieved context
String prompt = "Context information:\n" + context + "\n\nUser query: " + userQuery;
return chatModel.call(prompt);
}
}
This pattern allows for a clean separation between the retrieval component and the generation component in RAG applications.