High CPU Usage on Data Node

Elastic Stack Elasticsearch
1

Hi team ,

I have a linux server on which I am running elasticsearch datanode , Config of server is

48G of RAM , 1 TB HDD , 8 Core

. I am constantly having load of around 5-10

10:45:27 up 51 days, 17:54, 1 user, load average: 5.00, 6.04, 5.62

Please help me so that load can be made low .

Thanks & Regards
Arvind

2

Why do you want to lower i? Is it causing issues?

3

Seems due to this kibana is responding very slowly . and indexing performance is also low

4

If you're saying that load of 5-10 on an 8 core machine is causing really slow response, I would suggest adding another machine. If your load actually hovers closer to 5, which your average suggests, then moving to an SSD instead of HDD would be a good boost.

Main returns you're going to see are going to be from optimizing your mappings, and the way you index / query.

Taking a few shots in the dark here:

  • Make sure you're indexing in bulk instead of single documents
  • Increase refresh_interval for the index to a larger value of 60s or more.
  • Don't allocate more than 31GB to your elasticsearch heap. 32G+ requires 64-bit pointers, so you get less by giving your process more.
  • If reading disk indeed is a major bottleneck, try giving your elastic process 20G heap to leave more room for filesystem cache. (monitor with iostat)
  • Create your index with 7-8 shards instead of default 5, to spread load more across the cores if you are mainly indexing and have a few Kibana users to look at the data.

For more help please share mappings, sample of data, indexing and query patterns.