[2.1.3] Regarding Outputs to Multiple ES Clusters

Elastic Stack Logstash
1

Hi,
Atm, most of our traffic is hitting logstash via the beats input. We output to two clusters, one of which is significantly bigger than the other. And periodically, we see congestion reported by beats and it backs off.

My questions is, given this behaviour, do we need to split the output definition for our second elasticsearch cluster to a separate logstash instance? The idea being to avoid issues with one cluster impacting ingestion to the other...

Regards,
David

2

Logstash currently has a single event pipeline, so if you want to make outputs independent of each other you need two instances.

3

What version will we need to go to for this not to be an issue?

4

No currently released Logstash has support for multiple pipelines. I'm not sure it's even planned. Or am I misunderstanding the question?

5

You're correct.

6

Ok, thanks for clearing this up.