|
This version is still in development and is not considered stable yet. For the latest stable version, please use Spring AI 2.1.0! |
Metadata Filters
This section describes various filters that you can use against the results of a query.
Filter String
You can pass an SQL-like filter expression as a String to SearchRequest.Builder.filterExpression(String), to VectorStore.delete(String), or as the FILTER_EXPRESSION parameter of the RAG advisors.
similaritySearch has no String filter overload, so build a SearchRequest:
List<Document> results = vectorStore.similaritySearch(SearchRequest.builder()
.query("A story about a family")
.filterExpression("genre == 'drama' && year >= 2020")
.build());
Consider the following examples:
-
"country == 'BG'" -
"genre == 'drama' && year >= 2020" -
"genre in ['comedy', 'documentary', 'drama']" -
"genre not in ['comedy', 'documentary']" -
"(genre == 'drama' || genre == 'thriller') && year >= 2020" -
"not (year < 2020)" -
"author is not null"
The text syntax supports:
-
Comparisons:
==,!=,>,>=,<and⇐. -
List membership:
in […], andnin […]ornot in […]. -
Null checks:
is nullandis not null. -
Boolean operators:
&&(and),||(or) andnot, with parentheses for grouping. -
Values: strings in single or double quotes, integers, decimals and
trueorfalse. An integer with anLsuffix, such as1700000000000L, is aLong, and so is an integer too large for anint. -
Keys: plain identifiers, dotted keys such as
meta.country, and quoted keys such as'release-year'.
Keywords are accepted in upper or lower case, such as AND or and.
Filter.Expression
You can create an instance of Filter.Expression with a FilterExpressionBuilder that exposes a fluent API.
A simple example is as follows:
FilterExpressionBuilder b = new FilterExpressionBuilder();
Expression expression = b.eq("country", "BG").build();
You can build up sophisticated expressions by using the following operators:
EQUALS: '=='
GT: '>'
GE: '>='
LT: '<'
LE: '<='
NE: '!='
You can combine expressions by using the following operators:
AND: 'AND' | 'and' | '&&';
OR: 'OR' | 'or' | '||';
Considering the following example:
Expression exp = b.and(b.eq("genre", "drama"), b.gte("year", 2020)).build();
You can also use the following operators:
IN: 'IN' | 'in';
NIN: 'NIN' | 'nin' | 'NOT IN' | 'not in';
NOT: 'NOT' | 'not';
Consider the following example:
Expression exp = b.and(b.in("genre", "drama", "documentary"), b.not(b.lt("year", 2020))).build();
You can also use the following operators:
IS: 'IS' | 'is';
NULL: 'NULL' | 'null';
NOT NULL: 'NOT NULL' | 'not null';
Consider the following example:
Expression nullYear = b.isNull("year").build();
Expression withYear = b.isNotNull("year").build();
IS NULL and IS NOT NULL have not been implemented in all vector stores yet.
|
Use group() to keep a sub-expression together, as parentheses do in the text syntax:
Expression exp = b.and(
b.group(b.or(b.eq("genre", "drama"), b.eq("genre", "thriller"))),
b.gte("year", 2020)).build();
Partitioning a Shared Index
A single vector store is often shared by many logical groups of documents — for example one index holding content for several customers, projects, or knowledge bases.
A VectorStore operates over the whole index and returns or removes documents based on similarity and the filter you provide, so a filter expression is what selects the group you want to work with.
The approach is to tag each document with metadata that identifies its group when it is written, and then apply a matching filter on searches and deletes:
// When writing, record the group the document belongs to.
Document document = new Document(content, Map.of("group", groupId));
vectorStore.add(List.of(document));
// When searching, select documents from that group.
List<Document> results = vectorStore.similaritySearch(SearchRequest.builder()
.query(query)
.filterExpression("group == '" + groupId + "'")
.build());
// When deleting by filter, target the same group.
Filter.Expression scoped = new FilterExpressionBuilder().eq("group", groupId).build();
vectorStore.delete(scoped);
A couple of practices keep the groups cleanly separated:
-
Add the identifying metadata to every document at write time, so it is always available to filter on later.
-
Apply the corresponding filter consistently on searches and deletes so an operation only affects the intended group.