Map Files to specific Ingest Pipelines

Elastic Stack Beats
1

Hi,

I'm playing around with the new Elastic5 products and am interested in using the ingest node feature with Filebeat as the agent. I have a server that runs several applications with different log files/formats and it seems that per Filebeat agent you can only define a single Elasticsearch output that references a single pipeline.

It would be nice if it was possible to define a mapping of log files to output pipelines instead of having to maintain one big pipeline containing all of the grok patterns. I feel it could get messy trying to keep track of all of the patterns and which application's logs they are matching.

Something obvious I am missing or will I have to maintain a large list of grok patterns in one pipeline?

Thanks

2

You're not missing anything, it's hard to use when there are a lot of patterns at the moment. We plan to do something about this problem before we release 5.0.

I realized we don't have an issue for this in our repo yet. Could you please open one? Thanks

3

Thanks