# Nodes Space balancing Issue

**URL:** <https://discuss.elastic.co/t/nodes-space-balancing-issue/246378>\
**Category:** Elasticsearch\
**Created:** [August 26, 2020, 3:50am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378 "2020-08-26T03:50:33Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![shubhamblackstratus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubhamblackstratus/32/60703_2.png) [@shubhamblackstratus](https://discuss.elastic.co/u/shubhamblackstratus)\
**Post date:** [August 26, 2020, 3:50am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/1 "2020-08-26T03:50:33Z")

</div>

Hi Team,

I am having cluster which has three data nodes, which are having a single master. We found that due to some hardware failure, one of the data noes got stuck and shut down. We restarted that nodes and it joined back successfully.

But while checking shard allocation, I have found that that node was having 90% of space utilised even though it has low number of shards compared to other two.

As node 1 and 3 are showing 200GB spared for each and node 2 was shown less than 80GB. Even though node 1 and 3 are showing more allocated shards compared to node 2.

Please help me how to solve this situation.

Shubham

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 26, 2020, 3:52am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/2 "2020-08-26T03:52:00Z")

</div>

> [@shubhamblackstratus](#):
>
> single master

Do you mean only a single master eligible node out of all 3?

---

<div class="post-metadata">

**Author:** ![shubhamblackstratus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubhamblackstratus/32/60703_2.png) [@shubhamblackstratus](https://discuss.elastic.co/u/shubhamblackstratus)\
**Post date:** [August 26, 2020, 4:18am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/3 "2020-08-26T04:18:56Z")

</div>

No, Master node has been different node. I am having three data nodes and one separate master.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 26, 2020, 4:19am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/4 "2020-08-26T04:19:18Z")

</div>

Ok, but are the data nodes master eligible?

---

<div class="post-metadata">

**Author:** ![shubhamblackstratus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubhamblackstratus/32/60703_2.png) [@shubhamblackstratus](https://discuss.elastic.co/u/shubhamblackstratus)\
**Post date:** [August 26, 2020, 4:36am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/6 "2020-08-26T04:36:30Z")

</div>

One dedicated master node.  
the others are data only and master eligible.

My one of the data node got down and problem starts. Now, that node allocated more space by elasticsearch and the other two nodes are healthy. I got certain watermarks.

Strange thing is that, the node which has space issues, has low number of shards.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 26, 2020, 4:41am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/7 "2020-08-26T04:41:27Z")

</div>

What is the output from `_cat/allocation?v`.

---

<div class="post-metadata">

**Author:** ![shubhamblackstratus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubhamblackstratus/32/60703_2.png) [@shubhamblackstratus](https://discuss.elastic.co/u/shubhamblackstratus)\
**Post date:** [August 26, 2020, 4:45am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/8 "2020-08-26T04:45:16Z")

</div>

shards disk.indices disk.used disk.avail disk.total disk.percent host ip node  
266 731.2gb 775.4gb 62.3gb 837.7gb 92 x.x.x.x x.x.x.x enode2  
291 562.2gb 604.5gb 233.2gb 837.7gb 72 x.x.x.x x.x.x.x enode3  
291 562.2gb 605.8gb 231.9gb 837.7gb 72 x.x.x.x x.x.x.x enode1

---

<div class="post-metadata">

**Author:** ![shubhamblackstratus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shubhamblackstratus/32/60703_2.png) [@shubhamblackstratus](https://discuss.elastic.co/u/shubhamblackstratus)\
**Post date:** [August 27, 2020, 4:13am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/9 "2020-08-27T04:13:46Z")

</div>

Hey man, any help here ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 24, 2020, 4:13am UTC](https://discuss.elastic.co/t/nodes-space-balancing-issue/246378/10 "2020-09-24T04:13:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
