Best practice: single logstash instance with multiple pipelines or multiple logstash instances?

Elastic Stack Logstash
1

Hello everyone.

We are currently running a single Logstash instance which handles log collection from a number of different applications. There is approximately 10 pipelines, each with different filters for parsing the logs. The pipelines listen for beats, tcp and udp input.

We have had some unfortunate experiences where a faulty written filter in a single pipeline causes the entire Logstash instance to shut down. This behavior is of course something we do not want.

But what is the alternative? We are not sure it is beneficial to run 10 separate Logstash instances.

What is best-practice in this regard? Do we have a bad pattern since we are running this many pipelines in a single instance? Should we just do a better job of testing our pipelines locally before deploying to production?

2

I was wondering about this too.
Did you try with another logstash instance running with port 5045, then replicating the distributor pattern.
I am not too sure either, this is how i think i would solve this.
I would like to know the correct answer to your question too.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.