Full lesson
Explore the full explanation, examples, and visuals at your own pace.
Part 1
Configuring a Collector component doesn't turn it on. It takes two steps. First, you define its settings in a section, like receivers or exporters. Then you list it in a service pipeline. Only then does it run, and data actually flows. A component that's defined but never listed just sits there.
config.yaml: component sections
Here are the three sections. The otlp receiver is the data intake, listening on port 4317. The processors, memory_limiter and attributes, adjust data in between; attributes inserts environment equals production. The otlp_grpc exporter is the sender, pointed at otelcol:4317. Any setting you write overrides that component's default.
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
processors:
memory_limiter:
check_interval: 5s
limit_mib: 4000
attributes:
actions:
- key: environment
value: production
action: insert
exporters:
otlp_grpc:
endpoint: otelcol:4317config.yaml: service section
Defining components isn't enough. The service section switches them on. Under pipelines, a traces pipeline refers to each component by its identifier: otlp, memory_limiter, attributes, and otlp_grpc. Pipelines come in three types: traces, metrics, and logs. If a component isn't listed here, it stays off.
service:
pipelines:
traces:
receivers: [otlp]
processors: [memory_limiter, attributes]
exporters: [otlp_grpc]List order = processing order
Follow one span through the traces pipeline. The otlp receiver takes it in. Then memory_limiter checks it, attributes adds environment equals production, and otlp_grpc sends it on. Processors run in the order you list them, so swapping that list changes what happens to the span.
Predict the effect
General knowledgeTry the rules. Leave attributes out of the pipeline list, and spans never get environment=production. Reference a component you never defined, and the config fails. Reuse a processor in two pipelines, and each pipeline gets its own separate instance.
- Defined, not listed: never runs
- Listed, not defined: config error
- Shared processor: one instance per pipeline
type/name identifiers
What if you need two of the same component? Add a name after the type. Here, otlp/2 is a second otlp receiver, on its own port, feeding otlp_grpc/2, a second exporter. Together they form traces/2, a second traces pipeline. Each identifier just has to be unique.
Processors change data inside a pipeline. Connectors join two pipelines.
A processor works on data inside one pipeline. It can't hand traces over as metrics to another pipeline. That job needs a connector, which joins the two.
count connector: traces in, metrics out
The count connector is an exporter in the traces pipeline and a receiver in the metrics pipeline. Traces go in, and counts of matching span events come out, flowing to debug. The original traces aren't exported by this connector alone. To keep them, leave otlp_grpc in the traces exporters list too.
Extensions: beside the pipelines
Extensions sit beside the pipelines. They add Collector capabilities like health_check, pprof and zpages, which help with monitoring and diagnostics. They never touch telemetry data. The rule is the same: define them, then list them under service extensions.
- health_check, pprof, zpages
- No access to telemetry data
- Enable in service.extensions
Many sources, one final config
You can load config from several places: a file, an environment variable, an HTTPS address. Each one can be partial, and they merge in memory into one config. Then any --set overrides apply last. If the merged result is still incomplete, you get an error, because required components aren't added by default.
exporters: ${file:exporters.yaml} pulls one section from another file
A config can also pull in another file. Writing exporters with a file reference drops that file's contents into the section, and the Collector merges everything into one in-memory configuration.
Load, override, validate
Here, the Collector loads two sources: a file and an environment variable, and they merge. Then --set applies last, and double colons walk down nested keys, so exporters, debug, verbosity becomes detailed. Run otelcol validate to check the result before starting. And otelcol components lists what your distribution actually includes.
otelcol --config=file:base.yaml \
--config=env:MY_CONFIG_IN_AN_ENVVAR \
--set exporters::debug::verbosity=detailed
otelcol validate --config=file:base.yaml
otelcol componentsSections define components; service pipelines decide which ones actually run.
Sections define components; service pipelines decide which ones actually run. When data goes missing, check your service pipelines first, then run validate.
