Note: Vertex AI Search is being renamed to Agent Search. We are in the process of updating content to reflect the new branding.

Search with follow-ups

This page introduces search with follow-ups for Agent Search and shows you how to implement it using API calls.

If you want to add search with follow-ups to the search widget, see Configure results for the search widget.

Search with follow-ups applies to search apps with unstructured data and websites.

About search with follow-ups

Search with follow-ups is based on generative AI models. Search with follow-ups is different from the regular unstructured data search because search with follow-ups takes into account previous queries in the same search session.

Search with follow-ups supports the following:

Natural language query processing: It processes and understands human language input, identifies the intent behind a query, and returns relevant results.
Context awareness: It understands the context of prior interactions and provides context-aware answers.
Multi-turn: It allows users to ask follow-up questions and receive relevant responses.

Example of search with follow-ups

The following is an example of search with follow-ups. Suppose that you want to know about vacationing in Mexico:

You: When is the best time of the year to vacation in Mexico?
Search with follow-ups: The best time to vacation in Mexico is during the dry season, which runs from November to April.
You: What is the exchange rate?
Search with follow-ups: 1 USD is equal to approximately 17.65 Mexican pesos.
You: What's the average temperature in December?
Search with follow-ups: The average temperature varies from 70-78°F. Cancun's average is ~ 77°F.

With regular search, your question "What is the exchange rate"? wouldn't be answerable because regular search wouldn't know that you wanted the Mexican exchange rate. Similarly, a regular search wouldn't maintain context to give you temperatures for Mexico.

About conversations

In search with follow-ups, a conversation is made up of text queries provided by a user and responses provided by Agent Search.

These query and response pairs are sometimes referred to as messages. In the preceding example, the second message is made up of "What is the exchange rate?" and "1 USD is equal to approximately 17.65 Mexican pesos."

The conversations are stored in the same data store where the unstructured data is kept. In the data store, a conversation is represented by the Conversation resource. In addition to containing the query and response messages, the conversation resource has:

A unique name (the conversation ID).
A state (in-progress or completed).
A user pseudo ID, which is a visitor ID that tracks the user. It can be assigned programmatically.
A start time and an end time.

Before you begin

Make sure you satisfy the following prerequisites. Requirements vary depending on the type of app you have.

If you have an unstructured search app, make sure the Generative response option is turned on.
If you have a website search app, make sure the following are turned on:

Store conversations and get responses

You can use the command line or client libraries to generate search responses and to store the search-with-follow-ups conversation.

REST

To use the command line to create a conversation and generate responses from the user's input, follow these steps:

Specify the data store where you want to store the conversation history:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
"https://discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/conversations" \
-d '{
  "user_pseudo_id": "USER_PSEUDO_ID"
}'

PROJECT_ID: The project number or ID of your Google Cloud project
DATA_STORE_ID: The ID of the data store that is associated with your app.
USER_PSEUDO_ID: This is a unique identifier for tracking a search visitor. For example, you can implement this with an HTTP cookie, which uniquely identifies a visitor on a single device.

Click for an example response from the POST command.

{
  "name": "projects/12345/locations/global/collections/default_collection/dataStores/my-data-store_4321/conversations/11078281986791420687",
  "state": "IN_PROGRESS",
  "userPseudoId": "test_id",
  "startTime": "2023-08-15T20:08:12.094639Z"
}

Generate a search response and add it to a new or existing conversation in your data store:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json" \
"https://discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/global/collections/default_collection/dataStores/DATA_STORE_ID/conversations/CONVERSATION_ID:converse" \
-d '{
  "query": { "input": "FREE_TEXT"},
  "filter": "FILTER"
}'

PROJECT_ID: The project number or ID of your Google Cloud project
DATA_STORE_ID: The ID of the data store that is associated with your app.
CONVERSATION_ID: A unique ID for the conversation, for example, 123456. For a search-with-follow-ups conversation, use the same conversation ID in every turn.
FREE_TEXT: A free text string that contains the user's question—for example, what is bigquery?
FILTER: A text field for filtering search using a filter expression. The default value is an empty string. The way you construct your filter varies depending on whether you have website data or unstructured data with metadata. For more information, see Filter search with follow-ups.