Output specific input to specific pipelines

[dwjvaughan]

I have a filebeat on a server with multiple files to be ingested into a single Elasticsearch cluster. The issue I have is the files are very different, and as such, require very different ingest pipelines.

So the question is, how do I specify a different pipeline for a different input? Is it even possible?

I've just spotted this: Could filebeat support multipath for input and output separately? so I have a suspicion this won't be possible with a single instance of filebeat. Would appreciate confirmation either way though.

Many thanks,
Dave

[steffens (Steffen Siering)]

Filebeat 5.0 introduced the pipeline and pipelines settings. The pipeline configuration supports Format Strings allowing you to select the pipeline from event content itself (e.g. set via document_type or fields settings in prospector config).

[dwjvaughan]

Perfect, thank you.