Build a receiver

OpenTelemetry defines distributed tracing as:

Traces that track the progression of a single request, known as a trace, as it is handled by services that make up an application. The request may be initiated by a user or an application. Distributed tracing is a form of tracing that traverses process, network, and security boundaries.

Although distributed traces are defined in an application-centric way, you can think of them as a timeline for any request that moves through your system. Each distributed trace shows how long a request took from start to finish and breaks down the steps taken to complete it.

If your system generates tracing telemetry, you can configure your OpenTelemetry Collector with a trace receiver designed to receive and convert that telemetry. The receiver converts your data from its original format into the OpenTelemetry trace model so the Collector can process it.

To implement a trace receiver, you need the following:

• A Config implementation so the trace receiver can gather and validate its configurations in the Collector config.yaml.

• A receiver.Factory implementation so the Collector can properly instantiate the trace receiver component.

• A receiver.Traces implementation that collects the telemetry, converts it to the internal trace representation, and passes the telemetry to the next consumer in the pipeline.

This tutorial shows you how to create a trace receiver called tailtracer that simulates a pull operation and generates traces as an outcome of that operation.

Setting up receiver development and testing environment

First, use the Building a Custom Collector tutorial to create a Collector instance named otelcol-dev; all you need is to copy the builder-config.yaml described in Configure the OpenTelemetry Collector Builder and run the builder. As an outcome, you should now have a folder structure like this:

.
├── builder-config.yaml
├── ocb
└── otelcol-dev
    ├── components.go
    ├── components_test.go
    ├── go.mod
    ├── go.sum
    ├── main.go
    ├── main_others.go
    ├── main_windows.go
    └── otelcol-dev

To properly test your trace receiver, you may need a distributed tracing backend so the Collector can send the telemetry to it. We will be using Jaeger. If you don’t have a Jaeger instance running, you can easily start one using Docker with the following command:

docker run -d --name jaeger \
  -p 16686:16686 \
  -p 14317:4317 \
  -p 14318:4318 \
  jaegertracing/jaeger:latest

Once the container is up and running, you can access Jaeger UI via this URL: http://localhost:16686/

Now, create a Collector config file named config.yaml to set up the Collector components and pipelines.

touch config.yaml

For now, you just need a basic traces pipeline with the otlp receiver and the otlp and debug exporters. Here is what your config.yaml file should look like:

config.yaml

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

exporters:
  debug:
    verbosity: detailed
  otlp/jaeger:
    endpoint: localhost:14317
    tls:
      insecure: true
    sending_queue:
      batch:

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlp/jaeger, debug]
  telemetry:
    logs:
      level: debug

To verify that the Collector is properly set up, run this command:

./otelcol-dev/otelcol-dev --config config.yaml

The output may look like this:

2023-11-08T18:38:37.183+0800	info	service@v0.88.0/telemetry.go:84	Setting up own telemetry...
2023-11-08T18:38:37.185+0800	info	service@v0.88.0/telemetry.go:201	Serving Prometheus metrics	{"address": ":8888", "level": "Basic"}
2023-11-08T18:38:37.185+0800	debug	exporter@v0.88.0/exporter.go:273	Stable component.	{"kind": "exporter", "data_type": "traces", "name": "otlp/jaeger"}
2023-11-08T18:38:37.186+0800	info	exporter@v0.88.0/exporter.go:275	Development component. May change in the future.	{"kind": "exporter", "data_type": "traces", "name": "debug"}
2023-11-08T18:38:37.186+0800	debug	receiver@v0.88.0/receiver.go:294	Stable component.	{"kind": "receiver", "name": "otlp", "data_type": "traces"}
2023-11-08T18:38:37.186+0800	info	service@v0.88.0/service.go:143	Starting otelcol-dev...	{"Version": "1.0.0", "NumCPU": 10}

<OMITTED>

2023-11-08T18:38:37.189+0800	info	service@v0.88.0/service.go:169	Everything is ready. Begin running and processing data.
2023-11-08T18:38:37.189+0800	info	zapgrpc/zapgrpc.go:178	[core] [Server #3 ListenSocket #4] ListenSocket created	{"grpc_log": true}
2023-11-08T18:38:37.195+0800	info	zapgrpc/zapgrpc.go:178	[core] [Channel #1 SubChannel #2] Subchannel Connectivity change to READY	{"grpc_log": true}
2023-11-08T18:38:37.195+0800	info	zapgrpc/zapgrpc.go:178	[core] [pick-first-lb 0x140005efdd0] Received SubConn state update: 0x140005eff80, {ConnectivityState:READY ConnectionError:<nil>}	{"grpc_log": true}
2023-11-08T18:38:37.195+0800	info	zapgrpc/zapgrpc.go:178	[core] [Channel #1] Channel Connectivity change to READY	{"grpc_log": true}

If everything went well, the Collector instance should be up and running.

You may use the telemetrygen to further verify the setup. For example, open another console and run the following commands:

go install github.com/open-telemetry/opentelemetry-collector-contrib/cmd/telemetrygen@latest

telemetrygen traces --otlp-insecure --traces 1

You should be able to see detailed logs in the console and the traces in Jaeger UI via this URL: http://localhost:16686/.

Press Ctrl + C to stop the Collector instance in the Collector console.

Setting up Go module

Every Collector component should be created as a Go module. Let’s create a tailtracer folder to host our receiver project and initialize it as Go module.

mkdir tailtracer
cd tailtracer
go mod init github.com/open-telemetry/opentelemetry-tutorials/trace-receiver/tailtracer

It is recommended to enable Go Workspaces since we’re going to manage multiple Go modules: the otelcol-dev and tailtracer, and possibly more components over time.

cd ..
go work init
go work use otelcol-dev
go work use tailtracer

Designing and validating receiver settings

A receiver may have some configurable settings, which can be set via the Collector config file.

The tailtracer receiver will have the following settings:

• interval: a string representing the time interval (in minutes) between telemetry pull operations.
• number_of_traces: the number of mock traces generated for each interval.

Here is what the tailtracer receiver settings will look like:

receivers:
  tailtracer: # this line represents the ID of your receiver
    interval: 1m
    number_of_traces: 1

Create a file named config.go under the folder tailtracer where you will write all the code to support your receiver settings.

touch tailtracer/config.go

To implement the configuration aspects of a receiver, you need to create a Config struct. Add the following code to your config.go file:

package tailtracer

type Config struct{

}

To be able to give your receiver access to its settings, the Config struct must have a field for each of the receiver’s settings.

Here is what the config.go file should look like after you implemented the requirements above:

tailtracer/config.go

package tailtracer

// Config represents the receiver config settings in the Collector config.yaml
type Config struct {
   Interval    string `mapstructure:"interval"`
   NumberOfTraces int `mapstructure:"number_of_traces"`
}

Now that you have access to the settings, you can provide any kind of validation needed for those values by implementing the Validate method according to the optional ConfigValidator interface.

In this case, the interval value will be optional (we will look at generating default values later). But when defined, it should be at least 1 minute (1m) and the number_of_traces will be a mandatory value. Here is what the config.go looks like after implementing the Validate method:

tailtracer/config.go

package tailtracer

import (
	"fmt"
	"time"
)

// Config represents the receiver config settings in the Collector config.yaml
type Config struct {
	Interval