New FileBeat Version required new "Ingest Pipeline"

Elastic Stack Elasticsearch
1

hey,

So I updated filebeat on a couple machines and got an error:

"reason"=>"pipeline with id [filebeat-7.17.4-system-syslog-pipeline] does not exist"

After doing some digging, this was because I was previosuly using version 7.17.3.

So I had to manually create a new Ingest Pipeline with:

sudo filebeat setup -E output.logstash.enabled=false -E output.elasticsearch.hosts=['EMEA-ES-01.mydomain.com:9200'] -E setup.kibana.host=EMEA-kibana-01.mydomain.com:5601

Question:

Is there an easier/automated way to go about doing this when filebeat upgrades to a new version in the future?

1 Like
2

What we do is

  • We create an "ephemeral/container" filebeat instance (eg 7.17.4) with the version you are planning to install
  • This will complement existing ingest versions (eg 7.17.3). This way you can smoothly upgrade while agents from previous version is also sending data
  • Do the ingest pipeline setup in Elastic Cluster as a pre-task. Test with an agent before rolling out in PROD
  • The above is a one time task per Elastic Cluster per Filebeat version
2 Likes
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.