Query syntax

Vector queries operate by searching a vector database to find vectors that are best matched to your query vector. This page provides details about how this works.

Finding similar vectors

Vector search queries use two strategies:

  • K-Nearest Neighbors (KNN): Finds the k closest vectors to your query vector.
  • Approximate Nearest Neighbors (ANN): Finds the approximate k closest vectors to your query vector

To use KNN, indexes should be created with the FLAT vector index type. With KNN, search queries will be exact, but will be slower. To use ANN, indexes should be created with the HNSW vector index type. With ANN, search queries will be approximate, but will be faster. ANN accuracy can be improved by adjusting the HNSW index parameters and the EF_RUNTIME parameter in the query.

Query syntax breakdown

FT.SEARCH index "(hybrid_filter_expression)=>[KNN num_neighbours @my_vector_hash_key $my_vector_query_param]" PARAMS 2 my_vector_query_param "query_embedding" DIALECT 2

  • index: The name of the index containing your vector field.

  • (hybrid_filter_expression): This is the hybrid filter expression. Only tag and numeric indexes are supported in filter expressions. See Hybrid queries for more details on filter expressions.

    • (*) can be used to perform queries that don't require filtering.

  • =>: Separates the filter from the vector search.

  • [KNN num_neighbours @field $vector]: The KNN search expression. Replace num_neighbors with the chosen number of results and @field with your vector field's name.

  • PARAMS 2 my_vector_query_param "query_embedding":

    • The value 2 after PARAMS indicates that two additional arguments must be supplied.
    • my_vector_query_param is the query parameter vector name, as specified in the KNN search expression.
    • Replace query_embedding with your embedded query vector.

  • DIALECT 2: Specifies that you're using query dialect version 2 or later (required for vector search).

Hybrid queries

The initial expression enclosed within the parentheses () is a filter expression. Filter expressions allow you to filter vectors during execution of the vector search. A query that utilizes a filter expression to filter results is called a hybrid query. Any combination of tag and numeric indexes can form a hybrid query.

Memorystore for Redis uses two approaches for filtering vector searches:

  1. Pre-filtering: Pre-filtering relies on secondary indexes (e.g. tag, numeric) to first find the matches to the filter expression regardless of vector similarity. Once the filtered results are calculated a brute-force search is performed to sort by vector similarity.
  2. Inline-filtering: Inline-filtering performs the vector search algorithm (e.g. HNSW), ignoring found vectors which don't match the filter.

Pre-filtering is faster when the filtered search space is much smaller than the original search space. When the filtered search space is large, inline-filtering becomes faster. Memorystore for Redis automatically chooses between the two strategies based on the provided filter.

Filter expressions support both tag and numeric indexes.

Tag Index

Tags are text fields that are interpreted as a list of tags delimited by a separator character. Generally, tags are small sets of values with finite possible values like color, book genre, city name, or author.

  • Only indexed fields can be used as a tag filter.
  • TAG fields are tokenized by a separator character, which is a comma "," by default but configurable during index creation.
  • No stemming is performed while indexing a tag field.
  • Only prefix, exact pre filters can be performed on a tag field. Suffix, infix queries are not supported.
  • By default, tags are case insensitive. For example, "Blue" and "BLUE" both will be indexed as "blue" and will yield the same result in a hybrid query.
  • Empty strings are neither indexed or queried.