I'm trying to understand how to scale my ELK setup to a production deployment... But first some specs on my current system:
- Multiple Servers where I'm fetching data with Filebeat (around 10gb/day of logs)
- One central server with Elasticsearch, Kibana and Logstash with: 32GB RAM, 500GB memory disk, 4 processing cores;
So basically I just have one node on my system.
But in order to scale it up to A LOT of more servers I'm trying to understand what kind of hardware requirements I need, for:
- 100GB/day of log files
- Dashboards usage
- Retention period of two months
From the reading I've done all over the internet, I would have:
- 3 Master Nodes
- 3 Data Nodes, for storing all that data
Does this make any sense?? What kind of hardware requirements do I need in each node?? Also, in this case, each node means a server/VM, right??
Thanks a lot! All help would be highly appreciated... Just trying to understand this hardware area of the Elastic Stack, and it has not been easy!!