Tool Use

Language models are great at generating text, but often we need them to take real-world actions, such as querying an API, accessing a database, or calling a service. Most LLM providers support this through tool use (also known as function calling), where you expose specific operations in your application that the model can invoke.

Based on the input it receives, a model may choose to invoke (or call) one or more tools to augment its response. Your application then runs the corresponding logic for the tool using the parameters provided by the model. You then return the result to the model, allowing it to include the output in its final response.

The Toolkit simplifies tool integration by offering a structured, type-safe approach to defining tools. It takes care of all the wiring between the model and your application - all you have to do is define the tool and implement its behavior.

Defining a Tool

Let’s walk through a complete example of how to define, implement, and use a tool that fetches a dad joke from the icanhazdadjoke.com API.

1. Define the Tool

We start by defining a tool that the language model will have access to using the Tool.make constructor.

This constructor accepts several parameters that allow us to fully describe the tool to the language model:

• description: Provides an optional description of the tool
• success: The type of value the tool will return if it succeeds
• failure: The type of value the tool will return if it fails
• parameters: The parameters that the tool should be called with

Example (Defining a Tool)

1
import { Tool } from "@effect/ai"
2
import { Schema } from "effect"
3

4
const GetDadJoke = Tool.make("GetDadJoke", {
5
  description: "Get a hilarious dad joke from the ICanHazDadJoke API",
6
  success: Schema.String,
7
  failure: Schema.Never,
8
  parameters: {
9
    searchTerm: Schema.String.annotations({
10
      description: "The search term to use to find dad jokes",
11
    }),
12
  },
13
})

Based on the above, a request to call the GetDadJoke tool:

• Takes a single searchTerm parameter
• Will return a string if it succeeds (i.e. the joke)
• Does not have any expected failure scenarios

2. Create a Toolkit

Once we have a tool request defined, we can create a Toolkit, which is a collection of tools that the model will have access to.

Example (Creating a Toolkit)

1
import { Tool, Toolkit } from "@effect/ai"
2
import { Schema } from "effect"
3

4
const GetDadJoke = Tool.make("GetDadJoke", {
5
  description: "Get a hilarious dad joke from the ICanHazDadJoke API",
6
  success: Schema.String,
7
  failure: Schema.Never,
8
  parameters: {
9
    searchTerm: Schema.String.annotations({
10
      description: "The search term to use to find dad jokes",
11
    }),
12
  },
13
})
14

15
const DadJokeTools = Toolkit.make(GetDadJoke)

3. Implement the Logic

The .toLayer(...) method on a Toolkit allows you to define the handlers for each tool in the toolkit. Because .toLayer(...) takes an Effect, we can access services from our application to implement the tool call handlers.

Example (Implementing a Toolkit)

1
import { Tool, Toolkit } from "@effect/ai"
2
import {
3
  HttpClient,
4
  HttpClientRequest,
5
  HttpClientResponse,
6
} from "@effect/platform"
7
import { NodeHttpClient } from "@effect/platform-node"
8
import { Array, Effect, Schema } from "effect"