Is it still actual use Redis or Kafka before Elastic Stack?

I see many old articles where administrators reccomend use some buffer or broker before logstash or elasticsearch.
Many Linux Servers LOGS (Rsyslog) --> REDIS (Kafka etc.) --> logstash --> Elasticsearch --> Kibana
or simple
Many Linux Servers LOGS (Rsyslog) --> REDIS (Kafka etc.) --> Elasticsearch --> Kibana (without logstash)

Is it actual for new version of Logstash and Elasticsearch (5.5) or now native broker do it well? (Prevent data lose and load balance)


yes, it have great use of broker /buffer tool for log analysis and it is also with new version of Log stash and Elasticsearch.
you can use Redis or kafka or RabitMQ as buffer /broker and using of it all depends on your need.
Kafka and Redis. Both tools provide the functionality of data streaming and aggregation in their own respective ways.
A broker acts as a queue from which one or more Logstash instances can pull events. It' is also useful for load balancing requests and also acts as a buffer when there are lots of incoming events.

for more refer below link:

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.