I was wondering how the filebeat modules create and select the pipeline names.
I have these in my ES setup:
And I can see the content of the pipelines (https://github.com/elastic/beats/blob/master/filebeat/module/mysql/slowlog/ingest/pipeline.json), but I can't find where the given pipeline is selected when shipping data into Elasticsearch.
Filebeat has a concept of modules (
mysql, etc.). Each module has one or more filesets (e.g.
In each fileset's source code there is a
manifest.yml file. This file defines (amongst other things) the pipeline to be used for the fileset. Here are some examples:
Note that sometimes the pipeline might be chosen more dynamically based on some configuration value in the fileset's configuration. For example:
Hope that helps,
Hi @shaunak, thanks for the reply!
I see the file locations with the content of the pipelines, but I can't figure out where the pipeline names come from (filebeat-6.4.3-nginx-access-default, filebeat-6.4.3-mysql-slowlog-pipeline, etc...) and more importantly, how they are selected when Filebeat is shipping logs to Elasticsearch :).
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.