High Availability for Logstash

Anjan_Pratap · January 23, 2018, 7:47pm

Hi Everyone,

I have ELK configured with 4 ES Cluster Nodes , one logstash and one Kibana Node.
I want to have redundant logstash. Didnt find any solid procedure to configure logstash in cluster. I am using filebeat and kafka as input for my logstash.

Please suggest procedure on having multiple logstash nodes.

Thanks
Pratap

magnusbaeck · January 23, 2018, 9:21pm

So Logstash fetches all its input from Kafka? Make all Logstash instances consume from the same topic and consumer group.

Anjan_Pratap · January 24, 2018, 3:51pm

Thanks for the update,
I configured both the logstash instances to consume from the same kafka group.
If I stop logstash instance, the other logstash is processing the kafka messages.
But how can I manage fiebeat

Regards
Pratap.

magnusbaeck · January 25, 2018, 10:04am

But how can I manage fiebeat

Isn't Filebeat posting logs to Kafka?

Anjan_Pratap · January 25, 2018, 2:09pm

filebeat directly to logstash

ericrichter · January 25, 2018, 2:24pm

output.logstash:
hosts: ["host1:5044", "host2:5044"]
loadbalance: true

system · February 22, 2018, 2:25pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.