Kafka with elk stack

Elastic Stack Logstash
1

Hi,

When should we use kafka in ELK stack?
Where should we put kafka module in ELK stack provided filebeat is being used for log shipping?

2

FYI we’ve renamed ELK to the Elastic Stack, otherwise Beats feels left out :wink:

Usually Kafka is deployed as a broker layer, see https://www.elastic.co/guide/en/logstash/5.6/deploying-and-scaling.html#integrating-with-messaging-queues for some info.

3

are there some guides on how to deploy and configure with this architecture?
I already tried to use kafka but on my producer side, it does not send data to elasticsearch.

4

Kafka does not send to Elasticsearch anyway, you still need Logstash there to extract from it and then do the processing and sending.

5

Yes @warkolm, that's also what i did.
ES -> logstash -> kafka -> logstash -> es

6

Might b be easier to start another thread with more, specific details :slight_smile:

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.