Integrate fundamental AI concepts in Google Chat apps

1. Before you begin

What are Google Chat apps with AI?

Google Chat apps with AI do the following:

  • Bring your services and resources into Google Chat, which lets users get information and take action without leaving the conversation.
  • Integrate with generative AI models to create, search, and edit data like text or images.
  • Support an agentic experience by applying conversational AI concepts for more practical, natural, sophisticated and helpful interactions.

Why integrate Google Chat apps with AI?

The typical use cases fall in the following categories:

  • Content creation and edition. Generate marketing copy, craft social media posts, create realistic images, compose music, or aid in the creation of video content.
  • Data search and analysis. Extract key insights from an unstructured knowledge base, summarize lengthy texts, classify contents, or translate languages with enhanced accuracy and speed.
  • Conversation. Engage in natural, informative, and efficient conversations like you would with an assistant.
  • Task automation. Take actions on behalf of the user, such as creating a new calendar event, sending a document, or managing a ticket in an external system.

The ability to integrate these capabilities directly within the familiar interface of Google Chat is a huge opportunity for anyone who wants to improve the experience and productivity of their users.

Prerequisites

What you will build

In this codelab, you will build eight minimalist Google Chat apps that integrate fundamental AI concepts to show how they can be applied in real-world applications. They are all built as Google Workspace add ons and rely on the HTTP architecture:

2f1e2c66f6e2e2f0.png

It works as follows:

  1. A user sends a message in Google Chat to a Chat app, either as a direct message or in a Chat space.
  2. An HTTP request is sent to the web server running as a Node.js Google Cloud Run function that contains the Chat app logic.
  3. Optionally, the Chat app logic can integrate with Google Workspace services (like Calendar and Sheets), other Google services (like Maps, YouTube, and Vertex AI), or other web services (like a project management system or ticketing tool).
  4. The web server sends an HTTP response back to the Chat app service in Chat.
  5. The response is delivered to the user.
  6. Optionally, the Chat app can call the Chat API to asynchronously post messages or perform other operations.

Each Google Chat app's Node.js Google Cloud Run function contains their own version of the following source files to take the necessary actions in steps #3 and #6 above:

  • package.json: A central manifest that acts as a blueprint for the Node.js project. It's used to define the metadata, dependencies, and scripts.
  • env.js: A script that sets constants required for execution. It should be edited based on the environment and configuration.
  • index.js: The main script that handles the logic for the Google Chat interaction events. Only the message event type is implemented in this codelab but it would typically include other types such as card click, slash command, and dialog in real-life applications.

Prompt app

This app relies on a Gemini model to converse with users in their natural languages using concise and plain text answers.

5975b36968ab597a.gif

Format app

This app builds on the Prompt app by adding support for rich text answers compliant with the specific text format of Google Chat messages.

bc49e0acf0838f28.gif

Ground app

This app builds on the Format app by adding support for the Google Search tool and returning sources in answer messages with cards.

3cf232bf153f6abc.gif

MCP app

This app builds on the Format app by adding support for the Google Workspace Developer Assist Model Context Protocol (MCP).

8219c29366e9120e.gif

Multi-turn app

This app builds on the Format app by adding support for conversational memory with a Google Cloud Firestore database.

a819c274ce586451.gif

Custom tool app

This app builds on the Multi-turn app by adding support for a function calling custom tool that calls the Google Workspace Calendar API based on information provided by the user.

a1c4f586b7ab2e24.gif

Stream app

This app relies on a Gemini model to generate short stories based on themes provided by users. The Google Chat API is used to send results and statuses in messages as progress is made.

fd347ba03fe86e22.gif

Multimodal app

This app relies on a Gemini model to edit images based on textual instructions from users. Google Chat APIs are used to download and upload the images as message attachments.

57574be33474bbc.gif

What you will learn

  • The fundamental AI concepts are relevant to Google Chat apps and how to apply them.
  • To access Vertex AI using the Google Gen AI SDK.
  • To use Google Workspace APIs to develop delightful and powerful features.
  • To leverage Cloud Run to build scalable Google Chat apps.

What you will need

  • Completion of the Build an HTTP Google Chat app quickstart with Node.js. This codelab builds on the resulting Google Cloud project, Google Chat app, and Google Cloud Run function.

2. Get set up

Initialize and access resources

In this section, you access and configure the following resources from your preferred web browser.

Google Chat API configuration

Open the Google Cloud console in a new tab, then follow these steps:

  1. Select your project.
  2. In the Google Cloud search field, search for "Google Chat API", then click Google Chat API, click Manage, and click Configuration.

  1. Set the App name and Description to Gen AI App.
  2. Click Save.

9a06649cf9285b99.png

Google Chat space

Open Google Chat in a new tab, then follow these steps:

  1. If not already done, open a direct message space with the Chat app.
  2. Type Hello and press enter, the Chat app should reply with your name and avatar image.

e3b195c7b7b8e2af.png

Google Cloud Run function service

Open the Google Cloud console in a new tab, then follow these steps:

  1. Select your project.
  2. Click Menu ☰ > Cloud Run > Services.

  1. In the list of services, click addonchatapp, then open the Source tab.

b69df34ea0dc48a5.png

Download source code and resources locally

  1. Download this GitHub repository.

  1. In your preferred local development environment, open the node/chat/gen-ai-apps directory.

3c57c15db4ebfddb.png

3. Prompt app

This app prompts Gemini on Vertex AI to converse with users in their natural languages using concise and plain text answers. The implementation relies on the Google Gen AI SDK for Node.js.

Review concepts

Natural language

Any language spoken or written by humans for everyday communication, in contrast to artificial or computer-based languages.

Cloud Run functions

Cloud Run functions are great for building serverless backends, doing real-time data processing, and creating intelligent apps. There are no servers to provision, manage, patch, or update. They automatically scale, and are highly available and fault-tolerant.

Prompting

Prompting refers to the technique of crafting input (prompts) to guide a generative AI model to produce a desired output. It usually involves carefully phrasing questions, providing context, giving instructions, or giving examples to obtain specific and relevant responses from the model.

Vertex AI

Vertex AI offers everything you need to build and use generative AI, including AI solutions, search and conversation, more than 130 foundation models, and a unified AI platform.

c9e9c7a1945b22ac.png

Gemini

Gemini is a multimodal LLM from Google accessible through Vertex AI. It helps people unlock their human potential so that they can augment their imagination, expand their curiosity, and enhance their productivity.

Google Gen AI SDK

The Google Gen AI SDK is designed for developers to build applications powered by Gemini, it provides a unified interface compatible with both the Gemini Developer API and Vertex AI. It comes with client libraries in Python, Go, Node.js, and Java.

Review flow

c625fdcc8b4a27f4.png