Open Telemetry - flush failed (502)

Karol_Niema
Mon, 06 Oct 2025 12:27:13 +0000
Elastic Agent

Hello,

I am trying to use Elastic Agent in Open Telemetry collector mode,
after configure and started I receive error

2025-10-06T15:18:04.190+0300    error   elasticsearchexporter@v0.121.0/bulkindexer.go:346       bulk indexer flush error        {"otelcol.component.id": "elasticsearch/ecs", "otelcol.component.kind": "Exporter", "otelcol.signal": "metrics", "error": "flush failed (502): "}
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.flushBulkIndexer
        github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.121.0/bulkindexer.go:346
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*asyncBulkIndexerWorker).flush
        github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.121.0/bulkindexer.go:329
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.(*asyncBulkIndexerWorker).run
        github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.121.0/bulkindexer.go:322
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter.newAsyncBulkIndexer.func1
        github.com/open-telemetry/opentelemetry-collector-contrib/exporter/elasticsearchexporter@v0.121.0/bulkindexer.go:216

Configuration of otel.yml below

[****@**** Agent]$ cat otel.yml
receivers:
  # Receiver for platform specific log files
  filelog/platformlogs:
    include: [ /var/log/*.log ]
    retry_on_failure:
      enabled: true
    start_at: end
    storage: file_storage
#   start_at: beginning

  # Receiver for CPU, Disk, Memory, and Filesystem metrics
  hostmetrics/system:
    collection_interval: 30s
    scrapers:
      disk:
      filesystem:
      cpu:
        metrics:
          system.cpu.utilization:
            enabled: true
          system.cpu.logical.count:
            enabled: true
      memory:
        metrics:
          system.memory.utilization:
            enabled: true
      # process scraper is disabled for now: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/39423
      #process:
      #  mute_process_exe_error: true
      #  mute_process_io_error: true
      #  mute_process_user_error: true
      #  metrics:
      #    process.threads:
      #      enabled: true
      #    process.open_file_descriptors:
      #      enabled: true
      #    process.memory.utilization:
      #      enabled: true
      #    process.disk.operations:
      #      enabled: true
      network:
      processes:
      load:

extensions:
  file_storage:
    directory: /opt/Elastic/Agent/data/otelcol

processors:
  elasticinframetrics:
  resourcedetection:
    detectors: ["system"]
    system:
      hostname_sources: ["os"]
      resource_attributes:
        host.name:
          enabled: true
        host.id:
          enabled: false
        host.arch:
          enabled: true
        host.ip:
          enabled: true
        host.mac:
          enabled: true
        host.cpu.vendor.id:
          enabled: true
        host.cpu.family:
          enabled: true
        host.cpu.model.id:
          enabled: true
        host.cpu.model.name:
          enabled: true
        host.cpu.stepping:
          enabled: true
        host.cpu.cache.l2.size:
          enabled: true
        os.description:
          enabled: true
        os.type:
          enabled: true
  attributes/dataset:
    actions:
      - key: event.dataset
        from_attribute: data_stream.dataset
        action: upsert
  resource/process:
    attributes:
      - key: process.executable.name
        action: delete
      - key: process.executable.path
        action: delete

exporters:
  # Exporter to send logs and metrics to Elasticsearch
  elasticsearch/ecs:
    endpoints: ["http://******:9200,http://*******:9200,http://*******:9200"]
    api_key: *******
    mapping:
      mode: ecs
    logs_dynamic_index:
      enabled: true
    metrics_dynamic_index:
      enabled: true
    traces_dynamic_index:
      enabled: true
  elasticsearch/otel:
    endpoints: [ "http://*******:9200,http://******:9200,http://******:9200" ]
    api_key: **********
    mapping:
      mode: otel
    logs_dynamic_index:
      enabled: true
    metrics_dynamic_index:
      enabled: true
    traces_dynamic_index:
      enabled: true

service:
  extensions: [file_storage]
  pipelines:
    metrics/hostmetrics:
      receivers: [hostmetrics/system]
      processors: [elasticinframetrics, resourcedetection, attributes/dataset, resource/process]
      exporters: [elasticsearch/ecs]
    logs/platformlogs:
      receivers: [filelog/platformlogs]
      processors: [resourcedetection]
      exporters: [elasticsearch/otel]

I do not see any error on elastic side
I also checked api_key andlooks fine

curl -X GET "http://********:9200/_security/_authenticate" -H "Authorization: ApiKey *******==" -H "Content-Type: application/json"

{
    "username": "admin",
    "roles": [],
    "full_name": "Aministrator",
    "email": "*******",
    "metadata": {},
    "enabled": true,
    "authentication_realm": {
        "name": "_es_api_key",
        "type": "_es_api_key"
    },
    "lookup_realm": {
        "name": "_es_api_key",
        "type": "_es_api_key"
    },
    "authentication_type": "api_key",
    "api_key": {
        "name": "otel logs",
        "id": "cSNjuZkBDnv460pxuFid"
    }
}

2 posts - 1 participant