Filebeat modules

Elastic Stack Elasticsearch
1

We inherited a cluster and are trying to update the ingest pipeline (ES version 7.6)

Context: When we do GET ingest/pipeline there is a 15k line pipeline. It has all the processors from the filebeat modules they have uploaded: mysql,bro/zeek,suricata,aws,apache,azure etc. (they pretty much put in every module to provide for future expansion)

  1. We are wondering how can we add/adjust one specific module? If there are 5 modules loaded into one specific pipeline: How can we "PUT ingest/Pipeline" and make sure it only goes to the Mysql section of the pipeline?

  2. Is there a way to cat out all the pipelines that have been created? We run the GET ingest/pipeline and that returns the massive 15k pipeline but we are trying to see if there are other pipelines that have been created.

2

  1. You can setup a single module, yes. I don't know how Filebeat handles that in terms of ingest pipelines and loading everything though, which is the distinction you are looking for. It'd be worth creating a topic in #beats to ask more about that.

  2. Unfortunately only the way you have done it, you might be able to do some jq foo to filter things down though. Otherwise check out Cat endpoint for ingest pipelines · Issue #31954 · elastic/elasticsearch · GitHub and add a +1 reaction to the OP so can see what sort of demand there is for this.

I'd definitely recommend upgrading, 7.13 is latest and it's good to keep close to current if you can!

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.