# One node very high load average , why?

**URL:** https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236
**Category:** Elasticsearch
**Created:** [June 15, 2022, 8:14am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236 "2022-06-15T08:14:47Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![letwen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/letwen/32/107050_2.png) [@letwen](https://discuss.elastic.co/u/letwen)
#### Post date: [June 15, 2022, 8:14am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/1 "2022-06-15T08:14:48Z")

</div>

version6.8.0.one data node in cluster,16cpu 2.30GHz,48G memory,8G swap.have low cpu use,low memory use,but when i use top,can see very high 'load average',it be 40,sy high too,why ?

---

<div class="post-metadata">

### Author: ![letwen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/letwen/32/107050_2.png) [@letwen](https://discuss.elastic.co/u/letwen)
#### Post date: [June 15, 2022, 8:19am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/2 "2022-06-15T08:19:17Z")

</div>

when i restart this node scondday it just fine,norm 'load average',norm sy,norm cpu use,norm memory,cry myself

---

<div class="post-metadata">

### Author: ![letwen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/letwen/32/107050_2.png) [@letwen](https://discuss.elastic.co/u/letwen)
#### Post date: [June 15, 2022, 8:23am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/3 "2022-06-15T08:23:17Z")

</div>

and also,sometime data node well dead,so many non interruptible sleep state progress,and i can't use any like kill pkill ps command,force reboot is only i can do.

---

<div class="post-metadata">

### Author: ![linkerc](https://avatars.discourse-cdn.com/v4/letter/l/13edae/32.png) [@linkerc](https://discuss.elastic.co/u/linkerc)
#### Post date: [June 15, 2022, 6:07pm UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/4 "2022-06-15T18:07:09Z")

</div>

I have experienced similar thing before for a node running a long time.  
It seems to be running with higher cpu for no reason.  
Instead of rebooting the node, I simply restart the Elasticsearch service.  
Then the cpu utilization drops down to the level "I believe" to be normal.  
I came away with 2 possibilities:

1. Minor bug in ES. Restarting the service clears whatever state put the node in high CPU.
2. The short window when the service was restarting, some indices' primary shard got reassigned; therefore, the cluster becomes more balanced.

I'm leaning toward 1 because subsequently I had restarted several more nodes running more than 100 days and their CPU utilizations dropped as well. I couldn't remember which version it was.

But I have not seen such behavior with version 7.2 yet.

---

<div class="post-metadata">

### Author: ![letwen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/letwen/32/107050_2.png) [@letwen](https://discuss.elastic.co/u/letwen)
#### Post date: [June 16, 2022, 2:52am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/5 "2022-06-16T02:52:59Z")

</div>

thanks bro,but i can't change my version 😭

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 16, 2022, 6:56am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/6 "2022-06-16T06:56:58Z")

</div>

> [@letwen](#):
>
> 8G swap

You should always [run Elasticsearch without swap](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/setup-configuration-memory.html).

What type of storage do you have? Local SSDs? What does iostat look like?

---

<div class="post-metadata">

### Author: ![letwen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/letwen/32/107050_2.png) [@letwen](https://discuss.elastic.co/u/letwen)
#### Post date: [June 16, 2022, 8:45am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/7 "2022-06-16T08:45:12Z")

</div>

Is a customer provided virtual machine,but i'm sure not Local SSDs,i'm already configed bootstrap.memory\_lock: true，Will other programs read es cause this effect？

---

<div class="post-metadata">

### Author: ![linkerc](https://avatars.discourse-cdn.com/v4/letter/l/13edae/32.png) [@linkerc](https://discuss.elastic.co/u/linkerc)
#### Post date: [June 16, 2022, 5:33pm UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/8 "2022-06-16T17:33:17Z")

</div>

It's not that frequent for me back then. How long before you start to see this behavior? For me it's like 100+ days.  
What you could do if you truly believe you are experiencing the same issue as I is to schedule restart of the service say every 6 months (if manual restart is not possible), etc.

---

<div class="post-metadata">

### Author: ![letwen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/letwen/32/107050_2.png) [@letwen](https://discuss.elastic.co/u/letwen)
#### Post date: [June 17, 2022, 6:53am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/9 "2022-06-17T06:53:01Z")

</div>

I'll try it,thanks!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 15, 2022, 6:53am UTC](https://discuss.elastic.co/t/one-node-very-high-load-average-why/307236/10 "2022-07-15T06:53:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
