If I all my log generating modules can write into kafka why do i need logstash when kafka can directly write into elastic. Is there any disadvantage if I use this setup? One advantage I see is , in my understanding Logstash is not scalable and this setup will solve it.
logs--> kafka--> elastic.
Note: All my log sources can write into kafka.
Having kafka in front of logstash is a common use case. If you do not have any filters in logstash then you could indeed argue that it adds no value
Can the kafka do the data transforming part ( whatever logstash do with filter)?
It highly depends on your use case. One option is Kafka Streams (see here), but you should carefully evaluate your use case before deciding.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.