We decided to use ELK for our log analysis and i have been using it in my laptop for 3-4 weeks now and we do mostly visualizations for Apache and IIS web server logs.
We intend to take this to production and i need to come up with the hardware configuration.
The log data inputs are as follows.
- around 10-12 GB of log data is produced everyday
- We need to retain dashboard data for not more than 15 days.
- We don't do any specific search on elasticsearch just input log entries and we do charts out of them
we do avg response times , total hits and various pie charts , that's about it.
can you please give me some points in terms of memory , disk storage and cpu requirements.
UPDATE -- got an update saying we will be monitoring logs for around 50 apps each generating around 10 GB , so we are talking about processing 50 GB of data and the other requirements are the same.