Filebeat Modules

We inherited a cluster and are trying to update the ingest pipeline (ES version 7.6)

Context: When we do GET ingest/pipeline there is a 15k line pipeline. It has all the processors from the filebeat modules they have uploaded: mysql,bro/zeek,suricata,aws,apache,azure etc. (they pretty much put in every single module to provide for future expansion)

  1. We are wondering how can we add/adjust one specific module? If there are 5 modules loaded into one specific pipeline: How can we "PUT ingest/Pipeline" and make sure the changes only goes to the Mysql section of the pipeline?

  2. Is there a way to cat out all the pipelines that have been created? We run the GET ingest/pipeline and that returns the massive 15k pipeline but we are trying to see if there are other pipelines that have been created.

You're saying they combined all the module pipelines into one massive one? As for using the api to modify the pipeline, as far as I know it's all or nothing.

My 7.9.2 version of Kibana has "Ingest Node Pipelines" and shows about 70 different pipelines in my stack.

Also, you can do things like

GET _ingest/pipeline/*apache*

In my case, I get 2 filebeat apache pipelines

Correct, when you run Filebeat, it loads all the ingest pipelines for the modules. I guess I'm not understanding your question?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.