Dynamic pipeline selection with Kubernetes Autodiscover hints


I'm using Filebeat in Kubernetes with Autodiscover turned on. How can I route a specific container's logs to a specific ElasticSearch ingest pipeline, so that I can process the logs with the appropriate format grok pattern?

(Pier-Hugues Pellerin) #2

Hello @spiffytech

You could route events with specific labels to specific indices/pipeline.

  hosts: ["http://xxxxxxx:9200"]
  index: "filebeat-%{[beat.version]}-%{+yyyy.MM}"
  - index: "filebeat--%{[beat.version]}-%{+yyyy.MM}"
    pipeline: pihole-logging
      myfield: "myfieldvalue"

See https://www.elastic.co/guide/en/beats/filebeat/6.5/elasticsearch-output.html#indices-option-es for more details.

(system) closed #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.