Hello,
I am trying to use Elastic Agent in Open Telemetry collector mode,
after configure and started I receive error
2025-10-06T15:18:04.190+0300 error elasticsearchexporter@v0.121.0/bulkindexer.go:346 bulk indexer flush error {"otelcol.component.id": "elasticsearch/ecs", "otelcol.component.kind": "Exporter", "otelcol.signal": "metrics", "error": "flush failed (502): "}
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.flushBulkIndexer
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.121.0/bulkindexer.go:346
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*asyncBulkIndexerWorker).flush
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.121.0/bulkindexer.go:329
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*asyncBulkIndexerWorker).run
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.121.0/bulkindexer.go:322
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.newAsyncBulkIndexer.func1
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.121.0/bulkindexer.go:216
Configuration of otel.yml below
[****@**** Agent]$ cat otel.yml
receivers:
# Receiver for platform specific log files
filelog/platformlogs:
include: [ /var/log/*.log ]
retry_on_failure:
enabled: true
start_at: end
storage: file_storage
# start_at: beginning
# Receiver for CPU, Disk, Memory, and Filesystem metrics
hostmetrics/system:
collection_interval: 30s
scrapers:
disk:
filesystem:
cpu:
metrics:
system.cpu.utilization:
enabled: true
system.cpu.logical.count:
enabled: true
memory:
metrics:
system.memory.utilization:
enabled: true
# process scraper is disabled for now: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/39423
#process:
# mute_process_exe_error: true
# mute_process_io_error: true
# mute_process_user_error: true
# metrics:
# process.threads:
# enabled: true
# process.open_file_descriptors:
# enabled: true
# process.memory.utilization:
# enabled: true
# process.disk.operations:
# enabled: true
network:
processes:
load:
extensions:
file_storage:
directory: /opt/Elastic/Agent/data/otelcol
processors:
elasticinframetrics:
resourcedetection:
detectors: ["system"]
system:
hostname_sources: ["os"]
resource_attributes:
host.name:
enabled: true
host.id:
enabled: false
host.arch:
enabled: true
host.ip:
enabled: true
host.mac:
enabled: true
host.cpu.vendor.id:
enabled: true
host.cpu.family:
enabled: true
host.cpu.model.id:
enabled: true
host.cpu.model.name:
enabled: true
host.cpu.stepping:
enabled: true
host.cpu.cache.l2.size:
enabled: true
os.description:
enabled: true
os.type:
enabled: true
attributes/dataset:
actions:
- key: event.dataset
from_attribute: data_stream.dataset
action: upsert
resource/process:
attributes:
- key: process.executable.name
action: delete
- key: process.executable.path
action: delete
exporters:
# Exporter to send logs and metrics to Elasticsearch
elasticsearch/ecs:
endpoints: ["http://******:9200,http://*******:9200,http://*******:9200"]
api_key: *******
mapping:
mode: ecs
logs_dynamic_index:
enabled: true
metrics_dynamic_index:
enabled: true
traces_dynamic_index:
enabled: true
elasticsearch/otel:
endpoints: [ "http://*******:9200,http://******:9200,http://******:9200" ]
api_key: **********
mapping:
mode: otel
logs_dynamic_index:
enabled: true
metrics_dynamic_index:
enabled: true
traces_dynamic_index:
enabled: true
service:
extensions: [file_storage]
pipelines:
metrics/hostmetrics:
receivers: [hostmetrics/system]
processors: [elasticinframetrics, resourcedetection, attributes/dataset, resource/process]
exporters: [elasticsearch/ecs]
logs/platformlogs:
receivers: [filelog/platformlogs]
processors: [resourcedetection]
exporters: [elasticsearch/otel]
I do not see any error on elastic side
I also checked api_key andlooks fine
curl -X GET "http://********:9200/_security/_authenticate" -H "Authorization: ApiKey *******==" -H "Content-Type: application/json"
{
"username": "admin",
"roles": [],
"full_name": "Aministrator",
"email": "*******",
"metadata": {},
"enabled": true,
"authentication_realm": {
"name": "_es_api_key",
"type": "_es_api_key"
},
"lookup_realm": {
"name": "_es_api_key",
"type": "_es_api_key"
},
"authentication_type": "api_key",
"api_key": {
"name": "otel logs",
"id": "cSNjuZkBDnv460pxuFid"
}
}