Filebeat Modules

Elastic Stack Beats
1

We inherited a cluster and are trying to update the ingest pipeline (ES version 7.6)

Context: When we do GET ingest/pipeline there is a 15k line pipeline. It has all the processors from the filebeat modules they have uploaded: mysql,bro/zeek,suricata,aws,apache,azure etc. (they pretty much put in every single module to provide for future expansion)

  1. We are wondering how can we add/adjust one specific module? If there are 5 modules loaded into one specific pipeline: How can we "PUT ingest/Pipeline" and make sure the changes only goes to the Mysql section of the pipeline?

  2. Is there a way to cat out all the pipelines that have been created? We run the GET ingest/pipeline and that returns the massive 15k pipeline but we are trying to see if there are other pipelines that have been created.

2

You're saying they combined all the module pipelines into one massive one? As for using the api to modify the pipeline, as far as I know it's all or nothing.

3

My 7.9.2 version of Kibana has "Ingest Node Pipelines" and shows about 70 different pipelines in my stack.

Also, you can do things like

GET _ingest/pipeline/*apache*

In my case, I get 2 filebeat apache pipelines

4

Correct, when you run Filebeat, it loads all the ingest pipelines for the modules. I guess I'm not understanding your question?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.