Most people use the OpenTelemetry Collector as a glorified telemetry router.
But because sometimes need to go further that just built-in components, behind that façade, it is actually a pluggable, extensible framework designed for deep customization.
Let's see how we can use the Collector as a framework.
Why would I build a custom Collector?
No two observability stacks look the same.
You may need:
- A homegrown auth layer for incoming telemetry
- A data scrubber required by compliance
- An exporter for a system no one outside your company has ever heard of
- A receiver for a protocol that exists only in your engineering folklore
When the built-ins don’t cut it, the Collector gives you APIs to write the missing piece cleanly and officially.
What's in a Collector?
While using the Collector, you will interact with one (or more) of these component types:
Receivers
They ingest data and transform it into the internal data type to be consumed by other components.
Processors
They modify telemetry in-flight to do things such as sensitive data redation, enrichment, sampling, ...
Exporters
They take the processed data, possibly transform it and send it elsewhere (to another Collector, or to your observability provider).
Extensions
They are everything that's not telemetry: authentication, health checks, ...
Each component type follows the same pattern
- A config struct (your YAML settings)
- A factory (how the Collector creates your component)
- A component implementation (start, shutdown, and the processing logic)
Let’s build a component.
Your first processor
Let's build a minimal, but functional custom processor that adds an attribute to every span.
Here’s a minimal—but functional—custom processor that adds an attribute to every span.
There is already a built-in component, the attributesprocessor which can add attributes to a span.
Module Setup
The OpenTelemetry Collector is written in Go, and every component will plug into it.
So every component must be a Go module.
This post assumes you have basic knowledge of the Go programming language and how modules work.
You’ll need a Go module:
mkdir otel-processor-example
cd otel-processor-example
go mod init
Configuration struct
Every Collector component starts with a config.go file:
type Config struct {
AttributeKey string `mapstructure:"key"`
AttributeValue string `mapstructure:"value"`
}
This will let you configure the component like this:
processors:
addattribute:
key: environment
value: production
Processing logic
This is the actual logic that happend within your component.
It can start an HTTP server to receive requests (if it's a receiver), make outgoing HTTP requests (if it's an exporter), or more simply, take incoming data, transform it and pass it down the stack.
func createTracesProcessor(
ctx context.Context,
set processor.CreateSettings,
cfg component.Config,
) (processor.Traces, error) {
conf := cfg.(*Config)
return processorhelper.NewTracesProcessor(
ctx,
set,
conf,
func(ctx context.Context, td ptrace.Traces) (ptrace.Traces, error) {
rs := td.ResourceSpans()
for i := 0; i < rs.Len(); i++ {
ils := rs.At(i).ScopeSpans()
for j := 0; j < ils.Len(); j++ {
spans := ils.At(j).Spans()
for k := 0; k < spans.Len(); k++ {
span := spans.At(k)
span.Attributes().PutStr(conf.Key, conf.Value)
}
}
}
return td, nil
},
)
}
In this case, the processor walks down the trace data and adds an attribute to every span.
Factory
The factory wires everything together:
func NewFactory() component.ProcessorFactory {
return processor.NewFactory("addattribute", createDefaultConfig, processor.WithTraces(createTracesProcessor))
}
func createDefaultConfig() component.Config {
return &Config{
Key: "env",
Value: "default",
}
}
This tells the Collector that the component is called addattribute (so you can use that name in the config), and that it can handle traces.
Packaging a custom collector
Now that you have a custom component, you will want to use it.
In order to do that, you need to build a custom collector, and tell it to make this component available.
In a new folder (your custom collector is not the same thing as your component), create a builder-config.yaml file:
dist:
name: custom-collector
description: "Collector with custom processor"
output_path: ./dist
processors:
addattribute:
path: /path/to/your/component
Then, build your own Collector:
go install go.opentelemetry.io/collector/cmd/builder@latest
builder --config builder-config.yaml
You’ll get a custom Collector binary in the dist folder.
Running the collector
You can configure your new processor like any built-in one, by configuring config.yaml:
receivers:
otlp:
processors:
addattribute:
key: "environment"
value: "staging"
exporters:
logging:
service:
pipelines:
traces:
receivers: [otlp]
processors: [addattribute]
exporters: [logging]
Start the collector, send in some telemetry, and you’ll see your custom attribute added everywhere.
Wrapping Up
The OpenTelemetry Collector is more than a router. It’s an extensible engine where you can:
- Add new protocols
- Enforce custom security
- Transform data however you need
- Build exporters for any backend
- Share your components with the community
In fact, Elastic's Distribution of OpenTelemetry is a custom collector which includes both built-in components, as well as custom ones that are need to properly process data and send it to Elasticsearch.
You can dig through those custom components on the opentelemetry-collector-components repository.
Whether you want to scratch an itch in your own stack or contribute a future widely used component, building one is surprisingly approachable.