Develocity Reporting Kit Installation Manual

You are viewing the documentation for the Reporting Kit version 2.0. If you are upgrading from 1.x, please see Upgrading from 1.x to 2.x.

Introduction

This manual provides instructions for visualizing Develocity build data by installing the Develocity Reporting Kit into a Kubernetes cluster or onto a standalone host.

If you are looking to visualize Develocity build data in an AWS-hosted setup by using AWS-provided services, you can do that by using Amazon S3 and Amazon Athena.

The Develocity Reporting Kit (also known as the Reporting Kit) is a Kubernetes-based application, distributed as a Helm chart. Helm is a package manager for Kubernetes applications. Helm manages all Develocity Reporting Kit components.

The Reporting Kit is a companion application to a Develocity installation, and is only useful for existing Develocity customers or those trialling Develocity.

Build Data

A Build Scan® is a persisted record of a single build’s captured data. Each Build Scan consists of thousands to millions of very fine-grained events. Build models aggregate these events into higher-level structures to expose easily consumable, summarized information about the build.

Build models can be consumed via the Develocity API. This guide focuses on installing the Reporting Kit and connecting it to the API of an existing Develocity installation.

The builds endpoint of the Develocity API serves the build models that are available to the Reporting Kit.

Prerequisites

1. Develocity Installation

The Reporting Kit pulls build model data from an existing Develocity installation. To install the Reporting Kit you will need:

The URL of the Develocity installation. See Installing Into the Same Kubernetes Cluster as Develocity if deploying into the same Kubernetes cluster as Develocity.
Your Develocity license file.
An access key for an account on that instance with the necessary permissions.

The Reporting Kit can visualize Build Scan data from a Develocity instance of version 2023.4 or later. Some built-in dashboards rely on data available in later Develocity versions. See the compatibility table for details.

2. Build Environment

Most of the built-in dashboards rely on the presence of specific tags and custom values on Build Scan data to render the desired visualizations.

Tags used within the dashboards:

CI - Used throughout the dashboards to classify builds as either CI (tag is present) or local (tag is absent) builds

Custom values used within the dashboards:

Git repository - The URI of the Git repository that the build was run in, e.g., [email protected]:gradle/gradle.git
CI provider - The name of the CI provider that ran the build, e.g., GitHub Actions

If you are using the Common Custom User Data Gradle plugin, Common Custom User Data Maven extension or Common Custom User Data sbt plugin, this information will be automatically added to every Build Scan.

Host Requirements

This section outlines the requirements for installing the Develocity Reporting Kit.

1. Kubernetes Cluster

The Reporting Kit must be installed into a running Kubernetes cluster. The cluster you install the Reporting Kit into needs to have access to sufficient resources. Your cluster should be running a recent version of Kubernetes that’s still receiving patch support.

It’s also possible to install the Reporting Kit on a single node using the K3s lightweight Kubernetes distribution. Installation instructions are provided below.

We don’t recommend installing the Reporting Kit into the same standalone K3s cluster as Develocity, due to competition for resources. See the appendix if you are considering this.

2. Kubernetes Platforms

The Develocity Reporting Kit doesn’t use any platform specific features and is expected to work on all platforms, but we haven’t verified every available platform.

We’ve verified that the Reporting Kit works on K3s and Amazon EKS.

3. Helm Requirements

Check the Helm Version Support Policy to ensure compatibility with your Kubernetes version.

4. Resources

Node Group Specification

If you are planning to provision a dedicated cluster for your Develocity Reporting Kit installation, our recommended node group specification for that cluster is 2 nodes, each with 4 CPU units and 40 GiB memory.

Resource Requests and Limits

If you are planning to install the Reporting Kit in an existing cluster, we recommend ensuring access to at least 6 CPU units and 64 GiB memory.

The Develocity Reporting Kit Helm Chart’s total resource requests and limits are:

Resource requests (the minimal resources required by the application to start): 4.75 CPU units, 50.44 GiB memory.
Resource limits (the maximum resources that might be used by the application if available): 30.5 CPU units, 55.93 GiB memory.

Single-Node Installation

If you are planning to install the Develocity Reporting Kit on a single node, then that machine should meet these minimum requirements:

8 CPUs (x86_64 architecture)
64 GiB of memory

5. Storage

The Develocity Reporting Kit uses persistent volume claims for storing data. You can optionally provide the name of the desired storage class to be used for provisioning persistent volumes.

Some Pods are associated with persistent volumes and for Kubernetes platforms with multiple availability zones, the Pods and their persistent volumes must be located in the same zone. In this case it’s recommended to use a storage class with a volumeBindingMode of WaitForFirstConsumer to ensure that all persistent volumes are provisioned in the same zone that the Pod was scheduled in.

It’s strongly recommended to use storage classes that allow persistent volume claim expansion if available. This makes it straightforward to expand the storage used by the Reporting Kit.

Capacity Recommendations

The recommended minimum capacities for the persistent volumes are:

Description	Size in GiB
MinIO	100
Hive metastore	10

Description

Size in GiB

MinIO

100

Hive metastore

Exact MinIO storage requirements vary greatly depending on the number and size of Build Scans stored in your Develocity instance. We recommend monitoring the available space in your MinIO volume to ensure that your system doesn’t run out of space.

If your storage class doesn’t allow expanding volumes, you should also consider preparing for future data growth by adding additional disk capacity upfront.

Helm Configuration

The Develocity Reporting Kit is a Kubernetes-based application, distributed as a Helm chart. Helm is a package manager for Kubernetes applications, and it manages all Develocity Reporting Kit components. A Helm chart is a Kubernetes manifest template, with variables that can be provided at installation time.

Providing Configuration to Helm

Helm uses a values.yaml file to populate these variables and generate the Kubernetes manifests.

The variables in values.yaml configure the Develocity installation with information such as networking, database, or hostname settings.

Here is a minimal sample values.yaml file for installing the Develocity Reporting Kit:

values.yaml

develocity:
  address: https://develocity.example.com
  accessKey: "aecitwnpfw7h2sp3bl5uhrk5yedk47756obrsmneevvfe6jo2ssa"

Helm configuration can be provided in several ways:

Creating a Helm values file and passing it to helm using --values.
Passing values directly to the helm command using --set or --set-file.
Editing the default Helm values file in the chart before running helm.

We generally recommend setting values in a values.yaml file, because it means that your configuration is all in one place, and you have a more straightforward command to run for installation and upgrades.

Once your values.yaml file is complete, you will install the Reporting Kit using a command similar to the one below:

helm install --values ./values.yaml

Unless otherwise indicated, most values are optional and have usable defaults.

User-Managed Secrets

The Develocity Reporting Kit allows you to configure several secret values for various purposes described below. The Reporting Kit’s Helm chart allows you to set secret values directly in Helm configuration. In this case, Helm will create the Kubernetes Secret which contains the secret value. As an alternative, it’s usually possible in the Reporting Kit’s Helm chart to set the secret value in a Kubernetes Secret that you create and manage independently of Helm. If you use such a user-managed secret, you need to provide Helm with the name of the secret you have created for that purpose.

For example, if you want to configure the Grafana editor password using a user-managed secret, then you would create a secret (for example, using kubectl), take a note of the name, and then add it to your Helm values file using a prescribed Helm value.
For different secret values which the Reporting Kit’s Helm chart allows to be configured using a user-managed secret, you will need to use a different, specific Helm value, typically of the form some.thing.secretName. In this example, for the Grafana admin password, the Helm value to use is grafana.editorAccount.secretName. Wherever a user-managed secret can be used to configure a secret value, the data items that the secret needs to contain will be mentioned in the same place where the configuration of that secret value is documented.

Recommended Values

Here is a sample values.yaml file for installing the Develocity Reporting Kit:

values.yaml

# Target Develocity server
develocity:
  address: https://develocity.example.com
  accessKey: "aecitwnpfw7h2sp3bl5uhrk5yedk47756obrsmneevvfe6jo2ssa"

# Optionally create an editor account capable of creating custom dashboards
grafana:
  editorAccount:
    username: "editor"
    password: "showmethedata1234"

# Set the MinIO storage capacity a bit higher than the default
minio:
  storage:
    capacity: 200Gi

Deterministic Helm Template Output

If you use tools that redeploy applications based on changes in the output of helm template, such as ArgoCD, then you may need to ensure that the output of helm template is deterministic based on the Helm values you provide. To achieve this, read the appendix of this manual covering deterministic Helm output.

Helm Options

Each section below contains an overview of Develocity Reporting Kit installation options and their corresponding values.yaml variables:

Global options (license, images, annotations, storage class, security context, trusted certificates)
Develocity configuration
Pod resources
Access control
Ingress configuration
Scaling
Query results cache

1. Global Options

Configuration for Pulling Images Into the Installation Cluster

In order to pull images from the Gradle registry at https://registry.gradle.com/, you need to provide your Develocity license file to Helm at installation time when installing the Reporting Kit. This is the same license file as the one used for your Develocity installation. The easiest way to do this is to pass --set-file global.license.file=/path/to/develocity.license as an argument when running helm install.

You can also provide the license file inline in the values file. Only the "data" portion of the license is needed in the Helm value file, but it’s acceptable to include the entire license file contents:

values.yaml

global:
  license:
    file: R0VMRgF4nBWOSZKCMAAAX+QUu3BUIJAIwUQiwsViEwMIDKOyvH701n3p6nJBYxoRHnAUnwHwKLjb...

It’s also possible to specify the imagePullSecret using a user-managed secret.

Using a User-Managed Secret for Pulling Images

To manually create a secret containing the Develocity license within the same namespace as the Develocity Reporting Kit, follow these steps:

Create a namespace for the Develocity Reporting Kit, if it doesn’t already exist:

kubectl create namespace develocity-reporting-kit (1)

1	This example uses `develocity-reporting-kit` as the namespace, but it can be a custom name. If you use a custom name, update following commands accordingly.

Create a docker-registry secret with the Develocity license:

kubectl -n develocity-reporting-kit create secret docker-registry my-develocity-license-image-pull-secret \(1)
  --docker-username=develocity \
  "--docker-password=$(cat path/to/develocity.license)" \
  --docker-server=registry.gradle.com

1	`my-develocity-license-image-pull-secret` is the name of the secret. It can be any name you choose.

In your values.yaml file, be sure to include the name of the specific secret you want to use within the imagePullSecret key:

values.yaml

global:
  image:
    imagePullSecret: my-develocity-license-image-pull-secret

Airgap Installation Image Pull Policy

In a K3s-based airgap installation onto a standalone host, Helm should be configured so that no attempt is made to pull images from the outside world with the values below:

values.yaml