Elastic Stack with Kafka/Redis H/W requirement

Elastic Stack Elasticsearch
1

Hi All,

I want to deploy an Elastic Stack in a production environment. There are 1000+ client instances with around 15000/s logs. After reading various articles regarding this I have found these recommendation:

  1. 3x ElasticSearch Master/Data Nodes (2x 4 cores, 64GB, 10TB each for master & Data Nodes + 1x 2 cores 16GB only Master nodes)
  2. 2x Logstash Nodes (4 cores, 64GB each)
  3. 2x Kafka/Redis Nodes (4 cores, 64GB each)
  4. 1x Node for Kibana, Nginx (2 cores, 16GB)

Now, I want to divide two master and data nodes in separate physical machines. Another master node will reside in one of the previously mentioned machine. Same model will be applied for Logstash and Kafka/Redis. Is this okay for 15k/s logs?

As for buffer which one will to easy to implement and maintain? Redis or Kafka?

If you have any other recommendation please advise. Also thank you in advance.

2

Can anyone please give feedback regarding this? Need to know this urgently.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.