# Redis Vs Kafka Vs RabbitMQ in ELK

**URL:** https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212
**Category:** Logstash
**Created:** [July 11, 2016, 4:02pm UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212 "2016-07-11T16:02:08Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![kamal\_bisht](https://avatars.discourse-cdn.com/v4/letter/k/e47774/32.png) [@kamal\_bisht](https://discuss.elastic.co/u/kamal_bisht)
#### Post date: [July 11, 2016, 4:02pm UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212/1 "2016-07-11T16:02:08Z")

</div>

Hi

I am going to use ELK for my log analysis/monitoring. I need to use broker between logstash shipper and logstash indexer. Which tool Redis or Kafka or RabbitMQ I should use for this ?

Please help me on this.

---

<div class="post-metadata">

### Author: ![marke72](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marke72/32/10832_2.png) [@marke72](https://discuss.elastic.co/u/marke72)
#### Post date: [July 11, 2016, 4:50pm UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212/2 "2016-07-11T16:50:29Z")

</div>

I've never used Redis or Kafka but I am using RabbitMQ and I am liking it so far. But my cluster is small so I haven't seen how it performs with a large amount of data. It seems to work well with Logstash from what I've seen so far. I've never had problems with it except in the beginning just because of the learning curve.

---

<div class="post-metadata">

### Author: ![kamal\_bisht](https://avatars.discourse-cdn.com/v4/letter/k/e47774/32.png) [@kamal\_bisht](https://discuss.elastic.co/u/kamal_bisht)
#### Post date: [July 11, 2016, 5:07pm UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212/3 "2016-07-11T17:07:13Z")

</div>

Thanks, may I know how much data per day you are using for this..

---

<div class="post-metadata">

### Author: ![marke72](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marke72/32/10832_2.png) [@marke72](https://discuss.elastic.co/u/marke72)
#### Post date: [July 11, 2016, 5:22pm UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212/4 "2016-07-11T17:22:24Z")

</div>

At least 15 GB a day.

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 11, 2016, 8:13pm UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212/5 "2016-07-11T20:13:17Z")

</div>

If you edit your post and move it to the Logstash category you might get the attention from more people. The current category is for the deprecated logstash-forwarder tool but that's not what your question is about.

Either Redis, RabbitMQ, or Kafka will work for what you want to do. What's _best_ depends on your needs.

---

<div class="post-metadata">

### Author: ![kamal\_bisht](https://avatars.discourse-cdn.com/v4/letter/k/e47774/32.png) [@kamal\_bisht](https://discuss.elastic.co/u/kamal_bisht)
#### Post date: [July 11, 2016, 8:30pm UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212/6 "2016-07-11T20:30:04Z")

</div>

Hi

Can you please give some scenario where I can use Redis or Kafka or RabbitMQ

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [July 12, 2016, 5:35am UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212/7 "2016-07-12T05:35:49Z")

</div>

This is confusing. You say you want to use a broker and want advice on which one to pick, and then you follow up and request scenarios where a broker can be used.

A broker acts as a queue from which one or more Logstash instances can pull events. It's therefore useful for load balancing requests but also acts as a buffer when there are surges of incoming events (which is especially relevant when you have events arriving from sources that don't buffer locally themselves, i.e. if you can't receive their events they will be dropped on the floor).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:48am UTC](https://discuss.elastic.co/t/redis-vs-kafka-vs-rabbitmq-in-elk/55212/8 "2017-07-06T04:48:33Z")

</div>


