Output specific input to specific pipelines

Elastic Stack Beats
1

I have a filebeat on a server with multiple files to be ingested into a single Elasticsearch cluster. The issue I have is the files are very different, and as such, require very different ingest pipelines.

So the question is, how do I specify a different pipeline for a different input? Is it even possible?

I've just spotted this: Could filebeat support multipath for input and output separately? so I have a suspicion this won't be possible with a single instance of filebeat. Would appreciate confirmation either way though.

Many thanks,
Dave

2

Filebeat 5.0 introduced the pipeline and pipelines settings. The pipeline configuration supports Format Strings allowing you to select the pipeline from event content itself (e.g. set via document_type or fields settings in prospector config).

3

Perfect, thank you.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.