I am moving an ELK demo deployment (FileBeats -> Logstash cluster -> ElasticSearch cluster) to a production one. I am expecting ~12GB of daily logs to be fed into ElasticSearch, with a retention period of 1 year. I would like to ask about the sizing of the overall solution, and here is what I am thinking of:
- 2 x Logstash nodes (8 GB, 8 vCores each)
- 3 x ES Master nodes (6 GB, 4 vCores each)
- 3 x ES Data nodes (32 GB, 12 vCores each)
- Are the above adequate as a starting point? Can I safely reduce them?
- Do the specs of each node make sense for the expected amount of traffic? Are they over-sized?
- Do I need 3 ES Master nodes?
- Is a load-balancer required in front of ES nodes?