High Availability for Logstash

Elastic Stack Logstash
1

Hi Everyone,

I have ELK configured with 4 ES Cluster Nodes , one logstash and one Kibana Node.
I want to have redundant logstash. Didnt find any solid procedure to configure logstash in cluster. I am using filebeat and kafka as input for my logstash.

Please suggest procedure on having multiple logstash nodes.

Thanks
Pratap

2

So Logstash fetches all its input from Kafka? Make all Logstash instances consume from the same topic and consumer group.

3

Thanks for the update,
I configured both the logstash instances to consume from the same kafka group.
If I stop logstash instance, the other logstash is processing the kafka messages.
But how can I manage fiebeat

Regards
Pratap.

4

But how can I manage fiebeat

Isn't Filebeat posting logs to Kafka?

5

filebeat directly to logstash

6

output.logstash:
hosts: ["host1:5044", "host2:5044"]
loadbalance: true