Replace logstash with kafka

Elastic Stack Logstash
1

If I all my log generating modules can write into kafka why do i need logstash when kafka can directly write into elastic. Is there any disadvantage if I use this setup? One advantage I see is , in my understanding Logstash is not scalable and this setup will solve it.

logs--> kafka--> elastic.

Note: All my log sources can write into kafka.

2

Having kafka in front of logstash is a common use case. If you do not have any filters in logstash then you could indeed argue that it adds no value

3

Can the kafka do the data transforming part ( whatever logstash do with filter)?

4

It highly depends on your use case. One option is Kafka Streams (see here), but you should carefully evaluate your use case before deciding.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.