Is it still actual use Redis or Kafka before Elastic Stack?

Elastic Stack Elasticsearch
1

Hello.
I see many old articles where administrators reccomend use some buffer or broker before logstash or elasticsearch.
Like:
Many Linux Servers LOGS (Rsyslog) --> REDIS (Kafka etc.) --> logstash --> Elasticsearch --> Kibana
or simple
Many Linux Servers LOGS (Rsyslog) --> REDIS (Kafka etc.) --> Elasticsearch --> Kibana (without logstash)

Is it actual for new version of Logstash and Elasticsearch (5.5) or now native broker do it well? (Prevent data lose and load balance)

2

Hi,

yes, it have great use of broker /buffer tool for log analysis and it is also with new version of Log stash and Elasticsearch.
you can use Redis or kafka or RabitMQ as buffer /broker and using of it all depends on your need.
Kafka and Redis. Both tools provide the functionality of data streaming and aggregation in their own respective ways.
A broker acts as a queue from which one or more Logstash instances can pull events. It' is also useful for load balancing requests and also acts as a buffer when there are lots of incoming events.

for more refer below link:

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.