RabbtiMQ cluster with logstash cluster

Elastic Stack Logstash
1

Hi all,

I am quite sure that this a solved problem, so trying to figure out the optimal way...

My ELK stack is pretty basic, clients pushing message to RabbitMQ & Logstash pulls data (uses input plugin for RabbitMQ) from Queue and post it to elastic using ElasticSearch o/p plugin.

Now this works as expected, but now we would like fail over/resiliency to out environment for ex , if we box with rabbitMQ is down, other one should pick up aka RabbitMQ cluster,. Also just for availability/scalability side we would like to use, Logstash cluster that can pick up this RabbitMQ cluster and handle data for posting to Elastic DB.

I would need some help/recommendation on achieving steps above.

2

Logstash doesn't cluster like RabbitMQ or Elasticsearch, but that won't really be a problem in your case. Set up as many Logstash instances as you wish and configure their rabbitmq inputs to consume from the same RabbitMQ queue. The hostname they connect to should point to a load balancer that dispatches connections to an available server.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.