Multiple logstash instances reading the same files


(Marcin) #1

Hi guys!

My problem:
I have single NAS logs share connected to my application nodes, and logstash node. One logstash instance is reading logfiles, parsing and shipping to ES. Now, I want to add failover to logstash by deploying it to second node and connecting to the same files. Question is:

Can multiple logstash instances read the same files without duplicating whole traffic to ES?
If not, can I deploy two logstash instances in 'active-passive' mode?

I dont want to use beats*/logstash forwarder because there is no need for additional overhead, logs can be read directly from disk from any logstash node and this is faster and it simplifies ELK architecture, without additional points of failure.

-Thanks


(Christian Dahlqvist) #2

Logstash instances run independently and are not clustered, so I don't think it supports any active-passive setup with files as input. Why don't you monitor the Logstash process and restart it based on the same configuration if required?


(Marcin) #3

Thanks for reply. I'm already monitoring logstash instance, but just would like to know if there is any failover solution in my case.

So it seems that using beats on each app node and then multiple logstash nodes for shipping is the only case for redundancy.


(system) #4