# Kafka with elk stack

**URL:** <https://discuss.elastic.co/t/kafka-with-elk-stack/101430>\
**Category:** Logstash\
**Created:** [September 22, 2017, 6:51am UTC](https://discuss.elastic.co/t/kafka-with-elk-stack/101430 "2017-09-22T06:51:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jay\_Dabhi](https://avatars.discourse-cdn.com/v4/letter/j/8c91f0/32.png) [@Jay\_Dabhi](https://discuss.elastic.co/u/Jay_Dabhi)\
**Post date:** [September 22, 2017, 6:51am UTC](https://discuss.elastic.co/t/kafka-with-elk-stack/101430/1 "2017-09-22T06:51:54Z")

</div>

Hi,

When should we use kafka in ELK stack?  
Where should we put kafka module in ELK stack provided filebeat is being used for log shipping?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 22, 2017, 6:55am UTC](https://discuss.elastic.co/t/kafka-with-elk-stack/101430/2 "2017-09-22T06:55:55Z")

</div>

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats feels left out 😉

Usually Kafka is deployed as a broker layer, see [https://www.elastic.co/guide/en/logstash/5.6/deploying-and-scaling.html#integrating-with-messaging-queues](https://www.elastic.co/guide/en/logstash/5.6/deploying-and-scaling.html#integrating-with-messaging-queues) for some info.

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [September 22, 2017, 7:43am UTC](https://discuss.elastic.co/t/kafka-with-elk-stack/101430/3 "2017-09-22T07:43:40Z")

</div>

are there some guides on how to deploy and configure with this architecture?  
I already tried to use kafka but on my producer side, it does not send data to elasticsearch.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 22, 2017, 8:16am UTC](https://discuss.elastic.co/t/kafka-with-elk-stack/101430/4 "2017-09-22T08:16:57Z")

</div>

Kafka does not send to Elasticsearch anyway, you still need Logstash there to extract from it and then do the processing and sending.

---

<div class="post-metadata">

**Author:** ![jogoinar10](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jogoinar10/32/20068_2.png) [@jogoinar10](https://discuss.elastic.co/u/jogoinar10)\
**Post date:** [September 22, 2017, 8:47am UTC](https://discuss.elastic.co/t/kafka-with-elk-stack/101430/5 "2017-09-22T08:47:35Z")

</div>

Yes @warkolm, that's also what i did.  
ES -\> logstash -\> kafka -\> logstash -\> es

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 22, 2017, 8:48am UTC](https://discuss.elastic.co/t/kafka-with-elk-stack/101430/6 "2017-09-22T08:48:47Z")

</div>

Might b be easier to start another thread with more, specific details 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 20, 2017, 8:49am UTC](https://discuss.elastic.co/t/kafka-with-elk-stack/101430/7 "2017-10-20T08:49:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.