# Memory distribution on different nodes

**URL:** https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477
**Category:** Elasticsearch
**Created:** [June 24, 2020, 1:16pm UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477 "2020-06-24T13:16:19Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![rohitarorait82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohitarorait82/32/82981_2.png) [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)
#### Post date: [June 24, 2020, 1:16pm UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/1 "2020-06-24T13:16:19Z")

</div>

Hi All,

I am new to ELK cluster model and facing few issues with it and need your help on this. Currently elasticsearch is running on 6 different nodes but memory is getting used mostly in 2 nodes. Can someone suggest , How to distribute it equally

 ![Memory](https://us1.discourse-cdn.com/elastic/original/3X/5/7/57acbaa7985ebb988d4f8028aa588af05044fc45.png)

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 24, 2020, 10:18pm UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/2 "2020-06-24T22:18:34Z")

</div>

Can you provide the output of the `_cat/nodes?v` and `_cat/indices?v` as formatted text please?

---

<div class="post-metadata">

### Author: ![rohitarorait82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohitarorait82/32/82981_2.png) [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)
#### Post date: [June 25, 2020, 5:47am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/3 "2020-06-25T05:47:11Z")

</div>

Hi @warkolm ,

PFB output for both commands

/\_cat/indices?v

green open .security-7 PDrjVqc1QwmqqUJXk6VDww 1 1 42 0 180.3kb 90.1kb  
green open .reporting-2020.05.31 qFbU9qhjTqS7cQIWPv4RFQ 1 1 3 0 628.5kb 314.2kb  
green open .apm-custom-link \_ByqTv6BQYOt2dNKkt5dnw 1 1 0 0 416b 208b  
green open .reporting-2020.06.14 fIjpd4rlS0urUMPi3kT7UQ 1 1 1 0 174kb 103.7kb  
green open datapower\_visualization nwJK2o8IQRyoP-IJ24kSwg 1 1 2 0 49.8kb 24.9kb  
green open .kibana\_task\_manager\_1 dksbiM6-RvWSODaMq89VLw 1 1 5 1 83.4kb 30.6kb  
green open .apm-agent-configuration iwXeMFcLSG-SsZcQzxiloA 1 1 0 0 416b 208b  
green open .reporting-2020.06.07 IcXIe4DjT\_aUnvnlb8sd8w 1 1 2 0 669.7kb 334.8kb  
green open dox\_visualization bL3qSXnoR-Cx8HqvUjO9gQ 1 1 422300806 0 553.5gb 280.5gb  
green open .async-search yV2EemjQSEq5XhKO1GIUsA 1 1 56 3 43.9mb 21.9mb  
green open .kibana\_1 aQlc3MAeSbmjd5XrJNeAog 1 1 2623 17 1.2mb 616.9kb

/\_cat/nodes?v

ip heap.percent ram.percent cpu load\_1m load\_5m load\_15m node.role master name  
IP5 51 19 0 0.04 0.04 0.05 dilrt - datanode2  
IP4 31 14 0 0.00 0.01 0.05 dilmrt - datanode1  
IP1 20 16 2 0.28 0.28 0.30 lr - eai\_coordinatingnode  
IP3 64 15 0 0.02 0.03 0.05 dilmrt \* elasticsearch\_master  
IP2 58 74 6 0.31 0.57 0.65 dilmrt - masternode1  
IP6 47 94 3 0.23 0.34 0.32 dilrt - datanode3

 ![cat commands](https://us1.discourse-cdn.com/elastic/original/3X/d/5/d52e6e7219131af6d852ee96fc3aafe1fba82e7d.png)

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [June 25, 2020, 6:00am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/4 "2020-06-25T06:00:46Z")

</div>

Please don't post pictures of text, they are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them 🙂

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [June 25, 2020, 6:18am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/5 "2020-06-25T06:18:30Z")

</div>

You have one index with a single primary shard that contains almost all data in the cluster. As only one replica is configured there is only a total of 2 shards which are located on the nodes more heavily loaded. If you want better distribution you need to increase the number of primary shards, e.g. through the split index api, or create additional copies of the shard by increasing the number of replicas.

---

<div class="post-metadata">

### Author: ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)
#### Post date: [June 25, 2020, 6:27am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/6 "2020-06-25T06:27:49Z")

</div>

As @Christian_Dahlqvist said, you only have one big index. You can check were the shards are with  
`_cat/shards` . It would be no suprise to me if those shards are only on those 2 nodes.  
Here you can find the [Split Index API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-split-index.html). The new replicas should be distributed across the cluster.

---

<div class="post-metadata">

### Author: ![rohitarorait82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohitarorait82/32/82981_2.png) [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)
#### Post date: [June 25, 2020, 7:53am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/7 "2020-06-25T07:53:03Z")

</div>

Thanks @defalt and @Christian_Dahlqvist for you your suggestions 🙂

@warkolm I will take care of this in future, just pasted the pic for better visibility .

Few questions, Can I do runtime changes in shards using Dev Tools , Will increase in shards result in increase in Query time ?

Thanks in advance.

---

<div class="post-metadata">

### Author: ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)
#### Post date: [June 25, 2020, 7:58am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/8 "2020-06-25T07:58:05Z")

</div>

What do you mean by

> [@rohitarorait82](#):
>
> runtime changes in shards

?  
If you want to split the index you have to set it to read only. So you can't insert data while reindexing.  
Yes, the more shards you have the slower your search will be because you can only search one shard at a time. But it shouldnt make a significant difference.

---

<div class="post-metadata">

### Author: ![rohitarorait82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohitarorait82/32/82981_2.png) [@rohitarorait82](https://discuss.elastic.co/u/rohitarorait82)
#### Post date: [June 25, 2020, 9:31am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/9 "2020-06-25T09:31:06Z")

</div>

Thanks @defalt

Yes, i was asking the same, so there will be a downtime for this activity.

Also, since we are using coordinating node, it can take care of query search , I believe.

Thanks a lot for your help.

---

<div class="post-metadata">

### Author: ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)
#### Post date: [June 25, 2020, 10:25am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/10 "2020-06-25T10:25:37Z")

</div>

Awesome 👍. Please select on of the many answers as a solution so that others can find the answer more easily.  
🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 23, 2020, 10:25am UTC](https://discuss.elastic.co/t/memory-distribution-on-different-nodes/238477/11 "2020-07-23T10:25:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
