We have been having some heavy load on our system, but I am not exactly sure what causing them. Just wondering is there a way to check what have caused the high JVM load in our system. I know the high JVM load can be cause by high number of shards, number of requests , queries etc. But without a tangible figures, it will be hard to understand what exactly causing the high load.
We have 5 data nodes, and each nodes with 64GB RAM, and 12 cores CPU.
And we have 114 indices, and 988 shards spread across 5 data nodes.
We are thinking to add 2 more data nodes, and then reallocate some of the indices into those nodes.