# Replicas shards of Default indexers are in UNASSIGNED State

**URL:** <https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-monitoring\
**Created:** [June 27, 2023, 11:14am UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979 "2023-06-27T11:14:30Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sathish22](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@Sathish22](https://discuss.elastic.co/u/Sathish22)\
**Post date:** [June 27, 2023, 11:14am UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/1 "2023-06-27T11:14:30Z")

</div>

Hi Team,

Recently we are facing a issue with replicas shards and all default indexers replica shards are in UNASSIGNED State.

due to this cluster is going to yellow and red state.

> | index | shard | prirep | state |
> | --- | --- | --- | --- |
> | .tasks | 0 | r | UNASSIGNED |
> | .kibana-event-log-7.11.0-000019 | 0 | r | UNASSIGNED |
> | .monitoring-es-7-2023.06.25 | 0 | r | UNASSIGNED |
> | .ds-.logs-deprecation.elasticsearch-default-2022.07.21-000003 | 0 | r | UNASSIGNED |
> | .kibana\_task\_manager\_7.15.2\_001 | 0 | r | UNASSIGNED |

can any one please help me how to get this UNASSIGNED replica shared to ASSIGNED state or do we have a option to remove the replicas of default indexers.

Thanks,  
Sathish Thumma.

---

<div class="post-metadata">

**Author:** ![Sathish22](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@Sathish22](https://discuss.elastic.co/u/Sathish22)\
**Post date:** [July 2, 2023, 8:53am UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/2 "2023-07-02T08:53:59Z")

</div>

Can anyone, please give me some solution on above topic.

i have notices mainly below indexers replicas are not getting assigned

> 1. .ds-.logs-deprecation.elasticsearch-default \*\*\*\*
> 2. .kibana\*\*\*\*\*
> 3. .monitoring-es \*\*
> 4. .apm \*\*

> | index | shard | prirep | state | node | unassigned.reason |
> | --- | --- | --- | --- | --- | --- |
> | .apm-agent-configuration | 0 | r | UNASSIGNED | | REPLICA\_ADDED |
> | .kibana\_7.17.4\_reindex\_temp | 0 | r | UNASSIGNED | | CLUSTER\_RECOVERED |
> | .kibana\_7.15.0\_001 | 0 | r | UNASSIGNED | | REPLICA\_ADDED |
> | .async-search | 0 | r | UNASSIGNED | | REPLICA\_ADDED |
> | .monitoring-es-7-2023.06.27 | 0 | r | UNASSIGNED | | REPLICA\_ADDED |
> | .tasks | 0 | r | UNASSIGNED | | REPLICA\_ADDED |
> | .monitoring-es-7-2023.06.28 | 0 | r | UNASSIGNED | | REPLICA\_ADDED |
> | .ds-.logs-deprecation.elasticsearch-default-2022.06.23-000001 | 0 | r | UNASSIGNED | | CLUSTER\_RECOVERED |

Thanks,  
Sathish Thumma

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 2, 2023, 9:22am UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/3 "2023-07-02T09:22:46Z")

</div>

Which version of Elasticsearch are you using?

How many nodes do you have in your cluster?

How are these configured, e.g. with respect to different roles?

---

<div class="post-metadata">

**Author:** ![Sathish22](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@Sathish22](https://discuss.elastic.co/u/Sathish22)\
**Post date:** [July 2, 2023, 9:58am UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/4 "2023-07-02T09:58:57Z")

</div>

we are using version 7.17.10, and total 41 nodes(5 Master, 35 data), as per my know we didn't configured any roles

by using

> curl -X GET xxxxxxxxx:9200/\_cluster/allocation/explain?pretty" -H 'Content-Type: application/json' -d'  
> {  
> "index": ".kibana\_7.15.0\_001",  
> "shard": 0,  
> "primary": false  
> }  
> '

this i can able to see

> {  
> "index" : ".kibana\_7.15.0\_001",  
> "shard" : 0,  
> "primary" : false,  
> "current\_state" : "unassigned",  
> "unassigned\_info" : {  
> "reason" : "REPLICA\_ADDED",  
> "at" : "2023-06-26T17:12:19.599Z",  
> "last\_allocation\_status" : "no\_attempt"  
> },  
> "can\_allocate" : "no",  
> "allocate\_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",  
> "node\_allocation\_decisions" : [  
> {  
> "node\_id" : "xxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxx:xxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xx.xx"]"  
> }  
> ]  
> },  
> {

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [July 2, 2023, 12:57pm UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/5 "2023-07-02T12:57:01Z")

</div>

Please share the entire response for the allocation explain for that shard, you shared only part of it.

> [@Sathish22](#):
>
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xx.xx"]"

Also share the result of the following request: `GET /_cluster/settings`

---

<div class="post-metadata">

**Author:** ![Sathish22](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@Sathish22](https://discuss.elastic.co/u/Sathish22)\
**Post date:** [July 3, 2023, 2:25pm UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/6 "2023-07-03T14:25:17Z")

</div>

> {  
> "index" : ".kibana\_7.15.0\_001",  
> "shard" : 0,  
> "primary" : false,  
> "current\_state" : "unassigned",  
> "unassigned\_info" : {  
> "reason" : "REPLICA\_ADDED",  
> "at" : "2023-06-26T17:12:19.599Z",  
> "last\_allocation\_status" : "no\_attempt"  
> },  
> "can\_allocate" : "no",  
> "allocate\_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",  
> "node\_allocation\_decisions" : [  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156862464",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156932096",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156809216",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156813312",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156907520",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156862464",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156809216",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156739584",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "disk\_threshold",  
> "decision" : "NO",  
> "explanation" : "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%], actual free: [9.429708100306696%]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156907520",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156862464",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156903424",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156694528",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156874752",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156694528",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> },  
> {  
> "decider" : "disk\_threshold",  
> "decision" : "NO",  
> "explanation" : "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%], actual free: [9.342281105565426%]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156694528",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156932096",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156809216",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156932096",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156874752",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> },  
> {  
> "decider" : "disk\_threshold",  
> "decision" : "NO",  
> "explanation" : "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%], actual free: [8.509359847064598%]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156874752",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156903424",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156903424",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156739584",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "same\_shard",  
> "decision" : "NO",  
> "explanation" : "a copy of this shard is already allocated to this node [[.kibana\_7.15.0\_001][0], node[oFyaWzxdQHqEay6fG4d\_GQ], [P], s[STARTED], a[id=2XrjM3NeSAKTxkge\_MFdnQ]]"  
> },  
> {  
> "decider" : "disk\_threshold",  
> "decision" : "NO",  
> "explanation" : "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%], actual free: [9.914021061903375%]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156739584",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "disk\_threshold",  
> "decision" : "NO",  
> "explanation" : "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%], actual free: [8.838070635661516%]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156813312",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156895232",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156907520",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> },  
> {  
> "node\_id" : "xxxxxxxxxxxxxxxxxxxx",  
> "node\_name" : "xxxxxxxxxxxxxxxxxxxxxxx",  
> "transport\_address" : "xxxxxxxxx:xxxxxxxxx",  
> "node\_attributes" : {  
> "ml.machine\_memory" : "404156813312",  
> "ml.max\_open\_jobs" : "512",  
> "xpack.installed" : "true",  
> "ml.max\_jvm\_size" : "33766506496",  
> "transform.node" : "true"  
> },  
> "node\_decision" : "no",  
> "deciders" : [  
> {  
> "decider" : "filter",  
> "decision" : "NO",  
> "explanation" : "node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"  
> }  
> ]  
> }  
> ]  
> }

\_cluster/settings

> {  
> "persistent": {  
> "cluster": {  
> "routing": {  
> "allocation": {  
> "include": {  
> "\_ip": "xx.xx.xx.xxx"  
> },  
> "exclude": {  
> "\_ip": ""  
> }  
> }  
> }  
> },  
> "search": {  
> "max\_open\_scroll\_context": "5000"  
> },  
> "xpack": {  
> "monitoring": {  
> "collection": {  
> "enabled": "true"  
> }  
> }  
> }  
> },  
> "transient": {}  
> }

IP address which is in log

> node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"

is same in /\_cluster/settings

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [July 3, 2023, 3:44pm UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/7 "2023-07-03T15:44:59Z")

</div>

None of your nodes can allocate your shard because you are filtering their ip address and some that can allocate does not have enough space.

If you look at the response for the allocation explain you will see that the majority of your nodes can not allocate this shard because of this:

> node does not cluster setting [cluster.routing.allocation.include] filters [\_ip:"xx.xx.xxx.xx"]"

This means that the node IP address does not match the IP address in the include filter.

A couple of the nodes that match the ip being filtered cannot allocate because the disk already reached the low watermark:

> the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%], actual free: [9.914021061903375%]

And another one obvious cannot allocate because it already has the primary shard:

> a copy of this shard is already allocated to this node [[.kibana\_7.15.0\_001][0], node[oFyaWzxdQHqEay6fG4d\_GQ], [P], s[STARTED], a[id=2XrjM3NeSAKTxkge\_MFdnQ]]

---

<div class="post-metadata">

**Author:** ![Sathish22](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@Sathish22](https://discuss.elastic.co/u/Sathish22)\
**Post date:** [July 4, 2023, 10:44am UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/8 "2023-07-04T10:44:14Z")

</div>

Thanks a lot for understanding me what is the exact issue

for cluster.routing.allocation.include seeting is it ok to use below option

> PUT /\_cluster/settings  
> {  
> "persistent": {  
> "cluster.routing.allocation.include.\_ip": "\*"  
> }  
> }

and for disk watermark, can we use below commands to resolve this issue.

> cluster.routing.allocation.disk.threshold\_enabled: true
> 
> cluster.routing.allocation.disk.watermark.flood\_stage: 5gb
> 
> cluster.routing.allocation.disk.watermark.low: 30gb
> 
> cluster.routing.allocation.disk.watermark.high: 20gb

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 4, 2023, 10:50am UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/9 "2023-07-04T10:50:44Z")

</div>

> [@Sathish22](#):
>
> and for disk watermark, can we use below commands to resolve this issue.
> 
> > cluster.routing.allocation.disk.threshold\_enabled: true
> > 
> > cluster.routing.allocation.disk.watermark.flood\_stage: 5gb
> > 
> > cluster.routing.allocation.disk.watermark.low: 30gb
> > 
> > cluster.routing.allocation.disk.watermark.high: 20gb

What is the size of storage per node?

What is your average shard size?

---

<div class="post-metadata">

**Author:** ![Sathish22](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@Sathish22](https://discuss.elastic.co/u/Sathish22)\
**Post date:** [July 4, 2023, 2:09pm UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/10 "2023-07-04T14:09:53Z")

</div>

Storage per node = 4TB  
Average Shard size = 25GB

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 4, 2023, 2:16pm UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/11 "2023-07-04T14:16:04Z")

</div>

The default watermark is set at 15% of capacity, which would in your case be 600GB. Given that merging can cause shard sizes to almost double before disk space is reclaimed I think the values you provided are far too aggressive. I can see that you may want it lower than the default 600GB, but I would recommend lowering it to maybe half or a quarter of that, not to 30GB.

---

<div class="post-metadata">

**Author:** ![Sathish22](https://avatars.discourse-cdn.com/v4/letter/s/5daacb/32.png) [@Sathish22](https://discuss.elastic.co/u/Sathish22)\
**Post date:** [July 4, 2023, 5:42pm UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/12 "2023-07-04T17:42:45Z")

</div>

Thank you, for your suggestion, i will set low as 300GB,  
and can you please let me know, is there settings i need to make a part from this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 1, 2023, 5:43pm UTC](https://discuss.elastic.co/t/replicas-shards-of-default-indexers-are-in-unassigned-state/336979/13 "2023-08-01T17:43:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.