Grafana Cloud Enterprise Open source

Annotation and label template reference

Annotations and labels in alert rules can be defined using plain text. However, you can also define templates to customize their values with dynamic data from alert rule queries.

For example, you can template the summary annotation to include information from query values, providing relevant alert context for responders. Refer to Template annotations and labels for various use cases.

In templates, variables represent dynamic values from queries, while functions perform actions to transform or format this data.

Variables

Variables represent dynamic values from alert rule queries that can be displayed or accessed in your templates.

The $ and . symbols are used to reference variables and their properties. You can reference variables directly in your alert rule definitions using the $ symbol followed by the variable name. Similarly, you can access properties of variables using the dot (.) notation in alert rule templates.

{{ $values.A.Value }}

Templates are based on the Go templating system. Refer to Template language for additional information.

The following variables are available when templating annotations and labels:

VariablesDescription
$labelsContains all labels from the query, only query labels.
$valuesContains the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs.
$valueA string containing the labels and values of all instant queries; threshold, reduce and math expressions, and classic conditions in the alert rule. When a single data source is used, it returns the value of the query. It is generally recommended to use $values.

$labels

The $labels variable contains all labels from the query. It excludes user-configured and reserved labels, containing only query labels.

An alert rule displaying labels and value from a query.
An alert rule displaying labels and value from a query.

For example, suppose you have a query that returns CPU usage for all of your servers, and you have an alert rule that fires when any of your servers have exceeded 80% CPU usage for the last 5 minutes. You want to add a summary annotation to the alert that tells you which server is experiencing high CPU usage. With the $labels variable you can write a template that prints a human-readable sentence such as:

CPU usage for {{ $labels.instance }} has exceeded 80% for the last 5 minutes

The outcome of this template would be:

CPU usage for server1 has exceeded 80% for the last 5 minutes

If you are using a classic condition then $labels will not contain any labels from the query. Classic conditions discard these labels to enforce uni-dimensional behavior (at most one alert per alert rule). If you want to use labels from the query in your template then use the example here.

$values

The $values variable is a table containing the labels and floating point values of all instant queries and expressions, indexed by their Ref IDs (e.g, A, B, C, etc.). It does not contain the results of range queries, as they can return hundreds or thousands of rows.

Each Ref IDs, such as $values.A, has the following properties