Filebeat modules pipeline selection

I was wondering how the filebeat modules create and select the pipeline names.

I have these in my ES setup:

  • filebeat-6.4.3-nginx-access-default
  • filebeat-6.4.3-mysql-slowlog-pipeline
  • etc...

And I can see the content of the pipelines (, but I can't find where the given pipeline is selected when shipping data into Elasticsearch.

Filebeat has a concept of modules (nginx, mysql, etc.). Each module has one or more filesets (e.g. nginx/access, nginx/error, mysql/slowlog, etc.).

In each fileset's source code there is a manifest.yml file. This file defines (amongst other things) the pipeline to be used for the fileset. Here are some examples:

Note that sometimes the pipeline might be chosen more dynamically based on some configuration value in the fileset's configuration. For example:

Hope that helps,


Hi @shaunak, thanks for the reply!

I see the file locations with the content of the pipelines, but I can't figure out where the pipeline names come from (filebeat-6.4.3-nginx-access-default, filebeat-6.4.3-mysql-slowlog-pipeline, etc...) and more importantly, how they are selected when Filebeat is shipping logs to Elasticsearch :).

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.