Hi,
Is it possible to customise the naming of pipelines used by filebeat. Is there any example config?
Regards,
D
To use an Ingest Pipeline with Filebeat, you would first create that Ingest Pipeline in Elasticsearch and then reference it in your filebeat.yml configuration file, via the output.elasticsearch.pipeline setting. Since you create the Ingest Pipeline in Elasticsearch, you can name it whatever you want.
Read more about using Ingest Pipelines in Filebeat here: https://www.elastic.co/guide/en/beats/filebeat/current/configuring-ingest-node.html.
Ok thanks. Let's say I have a pipeline uploaded with a custom name and then decided to ship via logstash, how do I set the correct pipeline name in the filebeat metadata?
If you are shipping from Filebeat -> Logstash -> ES (running Ingest pipeline), why do you need to specify the Ingest pipeline name in Filebeat? Why not just specify it in the elasticsearch output configuration in your Logstash pipeline, via the pipeline setting: https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-pipeline?
That's what I'm doing via metadata @shaunak. So, I need to be sure that the pipeline metadata sent via filebeat matches the custom pipeline name in elasticsearch.
I'm not sure you can do it via the @metadata field. But you could add a custom field with your pipeline name in it using either the fields setting or the add_fields processor, and then reference it in your Logstash pipeline configuration.
Well, the index name can be specified so it's rather surprising that the pipeline name can't be overridden too.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.