Dynamic pipeline selection with Kubernetes Autodiscover hints

Elastic Stack Beats
1

I'm using Filebeat in Kubernetes with Autodiscover turned on. How can I route a specific container's logs to a specific ElasticSearch ingest pipeline, so that I can process the logs with the appropriate format grok pattern?

2

Hello @spiffytech

You could route events with specific labels to specific indices/pipeline.

output.elasticsearch:
  hosts: ["http://xxxxxxx:9200"]
  index: "filebeat-%{[beat.version]}-%{+yyyy.MM}"
  indices:
  - index: "filebeat--%{[beat.version]}-%{+yyyy.MM}"
    pipeline: pihole-logging
    when.is:
      myfield: "myfieldvalue"

See https://www.elastic.co/guide/en/beats/filebeat/6.5/elasticsearch-output.html#indices-option-es for more details.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.