We have a cluster with four elasticsearch nodes (v2.4), each having 4 cpu and 8gb of ram. At the moment, we have a load of about 500 messages a second.
Everytime we load a dashboard or use a query, the cpu usage is really heavy. Especially when increasing the timeframe. Well, that makes sense but I was wondering if it was normal with this cluster setup ? If yes, how much would we need to not have those heavy burning and lags on the web interface ?
Also, since elasticsearch can close indices which are several months old, and be reopened for long timeframe analysis (1 year for example), I was wondering if doing so would be dangerous for the cluster, since increasing the timeframe to 7 days is already puting all the CPU's to 100%.
Edit : I forgot to say that this is happening with only one user (being me) using the kibana web interface.
Would appreciate some insights on this.