Server suitability for ELK install

Elastic Stack Logstash
1

I have a single ELK server with the following hardware configuration:

  • Dual core Xeon 3.2Ghz
  • 20GB RAM
  • 500GB Hard disk space (RAID 1)

I have 10 remote hosts, generating a combined total of:

  • 16 GB of logs per day (Nginx and Java apps)
  • 50 million records per day
  • the bulk of the requests are spread throughout the business day (8am-7pm)

I'll need to keep approximately 2 weeks of logs before they can be purged. I'm aware that I should really be running an ES cluster for redundancy but there's no budget left for hardware.

Is the above server going to be capable of processing this log volume? Would other people mind sharing their ELK stats?

Thanks in advance for any replies.

2

You write that the bulk of requests are spread out during the day.. but how many users/applications will generate the requests? It's hard say without knowing the planned usage.

3

There are 10 servers sending requests via filebeat, and each server has 6 log files being monitored.

Having performed some analysis on the sum of all log files the average requests/sec is approximately 1000. I have performed some benchmarking on the server and am able to achieve an index rate of ~4000 per second so I think the hardware should be suitable.

4

Yes, it will probably be OK. Running more nodes is of course better if possible.