High Availability for Logstash

Anjan_Pratap · January 23, 2018, 7:47pm

Hi Everyone,

I have ELK configured with 4 ES Cluster Nodes , one logstash and one Kibana Node.
I want to have redundant logstash. Didnt find any solid procedure to configure logstash in cluster. I am using filebeat and kafka as input for my logstash.

Please suggest procedure on having multiple logstash nodes.

Thanks
Pratap

magnusbaeck · January 23, 2018, 9:21pm

So Logstash fetches all its input from Kafka? Make all Logstash instances consume from the same topic and consumer group.

Anjan_Pratap · January 24, 2018, 3:51pm

Thanks for the update,
I configured both the logstash instances to consume from the same kafka group.
If I stop logstash instance, the other logstash is processing the kafka messages.
But how can I manage fiebeat

Regards
Pratap.

magnusbaeck · January 25, 2018, 10:04am

But how can I manage fiebeat

Isn't Filebeat posting logs to Kafka?

Anjan_Pratap · January 25, 2018, 2:09pm

filebeat directly to logstash

ericrichter · January 25, 2018, 2:24pm

output.logstash:
hosts: ["host1:5044", "host2:5044"]
loadbalance: true

system · February 22, 2018, 2:25pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Cluster ( or High Avalibility) at Logstash level Logstash ccr-cross-cluster-replication	4	585	July 2, 2020
Using two kafka in the ELK setup(Load balancing) Logstash	4	897	October 24, 2017
How to achieve Clusters in Logstash? Logstash	6	1000	December 19, 2017
Multiple Logstash instances as Kafka inputs Logstash	1	1157	December 7, 2020
Multiple logstash servers need to be ache Beats filebeat	6	1283	November 17, 2016

High Availability for Logstash

Related topics