# Resolve unassigned shards? **URL:** \ **Category:** Elasticsearch\ **Tags:** ilm-index-lifecycle-management\ **Created:** [June 23, 2021, 1:16pm UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793 "2021-06-23T13:16:52Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![lchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lchan/32/90857_2.png) [@lchan](https://discuss.elastic.co/u/lchan)\ **Post date:** [June 23, 2021, 1:16pm UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793/1 "2021-06-23T13:16:53Z")
I am an unassigned shards issue on a newly setup cluster where the ilm policy is not moving the shares from hot to warm. The ilm policy is set to rollover after 1d or 50gb but the allocation is complaining about node does not match "node does not match index setting". ```auto ... "phases" : { "warm" : { "min_age" : "7d", "actions" : { "set_priority" : { "priority" : 50 } } } ... "hot" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_primary_shard_size" : "50gb", "max_age" : "1d" }, "set_priority" : { "priority" : 100 } } } ``` ```auto # GET /_cat/health?v epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 1624454091 13:14:51 14e1656e7d5444eb8cc308b30ae1418a yellow 5 4 76 76 0 0 1 0 - 98.7% ``` ```auto # GET /_cluster/allocation/explain?pretty { "index" : "filebeat-7.12.1-2021.06.23-000217", "shard" : 0, "primary" : false, "current_state" : "unassigned", "unassigned_info" : { "reason" : "INDEX_CREATED", "at" : "2021-06-23T09:34:10.741Z", "last_allocation_status" : "no_attempt" }, "can_allocate" : "no", "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes", "node_allocation_decisions" : [ { "node_id" : "oV7Ky2keQyyFODvx5MHMuQ", "node_name" : "instance-0000000009", "transport_address" : "10.46.88.30:19523", "node_attributes" : { "logical_availability_zone" : "zone-0", "server_name" : "instance-0000000009.14e1656e7d5444eb8cc308b30ae1418a", "availability_zone" : "westus2-3", "xpack.installed" : "true", "data" : "cold", "instance_configuration" : "azure.data.highstorage.e16sv3", "transform.node" : "false", "region" : "unknown-region" }, "node_decision" : "no", "weight_ranking" : 1, "deciders" : [ { "decider" : "filter", "decision" : "NO", "explanation" : """node does not match index setting [index.routing.allocation.require] filters [data:"hot"]""" } ] }, { "node_id" : "wLGNkgAYQ7yqQgUtedIiXw", "node_name" : "instance-0000000010", "transport_address" : "10.46.88.61:19471", "node_attributes" : { "logical_availability_zone" : "zone-0", "server_name" : "instance-0000000010.14e1656e7d5444eb8cc308b30ae1418a", "availability_zone" : "westus2-3", "xpack.installed" : "true", "data" : "frozen", "instance_configuration" : "azure.es.datafrozen.lsv2", "transform.node" : "false", "region" : "unknown-region" }, "node_decision" : "no", "weight_ranking" : 2, "deciders" : [ { "decider" : "filter", "decision" : "NO", "explanation" : """node does not match index setting [index.routing.allocation.require] filters [data:"hot"]""" }, { "decider" : "disk_threshold", "decision" : "NO", "explanation" : "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], using more disk space than the maximum allowed [85.0%], actual free: [9.980344772338867%]" }, { "decider" : "dedicated_frozen_node", "decision" : "NO", "explanation" : "this node's data roles are exactly [data_frozen] so it may only hold shards from frozen searchable snapshots, but this index is not a frozen searchable snapshot" } ] }, { "node_id" : "NBchKHq-R-O2qp_XrYDHCQ", "node_name" : "instance-0000000007", "transport_address" : "10.46.88.23:19757", "node_attributes" : { "logical_availability_zone" : "zone-0", "server_name" : "instance-0000000007.14e1656e7d5444eb8cc308b30ae1418a", "availability_zone" : "westus2-3", "xpack.installed" : "true", "data" : "warm", "instance_configuration" : "azure.data.highstorage.e16sv3", "transform.node" : "false", "region" : "unknown-region" }, "node_decision" : "no", "weight_ranking" : 3, "deciders" : [ { "decider" : "filter", "decision" : "NO", "explanation" : """node does not match index setting [index.routing.allocation.require] filters [data:"hot"]""" } ] }, { "node_id" : "6la3uCyRQaOTDo55VQOOEw", "node_name" : "instance-0000000012", "transport_address" : "10.46.88.125:19791", "node_attributes" : { "logical_availability_zone" : "zone-0", "server_name" : "instance-0000000012.14e1656e7d5444eb8cc308b30ae1418a", "availability_zone" : "westus2-1", "xpack.installed" : "true", "data" : "hot", "instance_configuration" : "azure.data.highio.l32sv2", "transform.node" : "true", "region" : "unknown-region" }, "node_decision" : "no", "weight_ranking" : 4, "deciders" : [ { "decider" : "same_shard", "decision" : "NO", "explanation" : "a copy of this shard is already allocated to this node [[filebeat-7.12.1-2021.06.23-000217][0], node[6la3uCyRQaOTDo55VQOOEw], [P], s[STARTED], a[id=eFlQVZQ0QqKnphk43b7b8Q]]" } ] } ] } ``` ```auto # GET filebeat-7.12.1-2021.06.23-000217/_ilm/explain { "indices" : { "filebeat-7.12.1-2021.06.23-000217" : { "index" : "filebeat-7.12.1-2021.06.23-000217", "managed" : true, "policy" : "filebeat", "lifecycle_date_millis" : 1624440850656, "age" : "3.63h", "phase" : "hot", "phase_time_millis" : 1624440850976, "action" : "rollover", "action_time_millis" : 1624440851432, "step" : "check-rollover-ready", "step_time_millis" : 1624440851432, "phase_execution" : { "policy" : "filebeat", "phase_definition" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_primary_shard_size" : "50gb", "max_age" : "1d" }, "set_priority" : { "priority" : 100 } } }, "version" : 16, "modified_date_in_millis" : 1623638569137 } } } } ``` Someone has similar issue but no response... > [@Cannot allocate because allocation is not permitted to any of the nodes](https://discuss.elastic.co/t/cannot-allocate-because-allocation-is-not-permitted-to-any-of-the-nodes/244530): > > Hi , we are facing issue on elasticsearch, when we are adding replica 1 and shrads 1 on templates then iml policy is not working properly. indices are not moving from warm to cold phase and delete phase. [image] { "index" : "cat-2020.08.11-000001", "shard" : 0, "primary" : false, "current\_state" : "unassigned", "unassigned\_info" : { "reason" : "INDEX\_CREATED", "at" : "2020-08-11T09:57:06.687Z", "last\_allocation\_status" : "no\_attempt" }, "can\_allocate" : "no", "allocate\_explanation… ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 24, 2021, 5:47am UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793/2 "2021-06-24T05:47:19Z")
What's the output from `_cat/nodes?v`? ---
**Author:** ![lchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lchan/32/90857_2.png) [@lchan](https://discuss.elastic.co/u/lchan)\ **Post date:** [June 25, 2021, 6:36pm UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793/3 "2021-06-25T18:36:09Z")
```auto ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name 10.46.88.125 28 100 16 8.16 9.12 8.39 himrst * instance-0000000012 10.46.88.30 61 86 19 13.27 9.39 8.33 cr - instance-0000000009 10.46.88.23 40 95 3 0.65 0.95 1.17 rw - instance-0000000007 10.46.88.61 63 94 2 0.81 0.66 0.68 f - instance-0000000010 10.46.88.37 60 90 2 1.98 2.17 2.10 lr - instance-0000000013 ``` ---
**Author:** ![ropc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ropc/32/47022_2.png) [@ropc](https://discuss.elastic.co/u/ropc)\ **Post date:** [June 26, 2021, 1:29am UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793/4 "2021-06-26T01:29:11Z")
Hi @lchan - I checked your deployment and noticed that you have an active subscription with us. You have been in touch with our support team and it seems that the problem has been resolved. As discussed with our team, `filebeat-7.12.1-2021.06.23-000217` replica shard could not be assigned in the hot phase because there is only one hot node in your deployment. The same copy of a shard cannot be allocated in the same node. Feel free to engage with our support team at any time, it is one of the perks in your subscription 🙂 ---
**Author:** ![lchan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lchan/32/90857_2.png) [@lchan](https://discuss.elastic.co/u/lchan)\ **Post date:** [June 26, 2021, 2:49am UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793/5 "2021-06-26T02:49:32Z")
Hi @ropc, Sometimes I post in here so other users can take advantage. Thanks for taking a step of above and beyond for looking out on this issue (By the way, how do you find out my deployment ID? By ip address? Even when I open a support ticket, I will be asked for deployment ID if I forgot to include one) Yes The issue(s) was two folded. The unassigned shards DO NOT affect ILM which I thought it would. - Unassigned shards - Were cause by the default number of replica of 1. - The index has a data:hot allocation requirement on "index.routing.allocation.require.data" from a legacy template on . **cloud-hot-warm-allocation-0** and filtering indices. > **[Migrate index allocation filters to node roles | Elasticsearch Guide \[7.13\] |...](https://www.elastic.co/guide/en/elasticsearch/reference/current/migrate-index-allocation-filters.html#stop-setting-custom-hot-attribute)** ---
**Author:** ![ropc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ropc/32/47022_2.png) [@ropc](https://discuss.elastic.co/u/ropc)\ **Post date:** [June 26, 2021, 3:17am UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793/6 "2021-06-26T03:17:57Z")
> Sometimes I post in here so other users can take advantage. Thank you for being active in our community. I love the spirit! 🙂 > Thanks for taking a step of above and beyond for looking out on this issue (By the way, how do you find out my deployment ID? By ip address? The cluster ID was in the result of the `_cat/health` provided earlier. I used this information to correlate your deployment in our backend systems. > Even when I open a support ticket, I will be asked for deployment ID if I forgot to include one) Correct - the support team may require confirmation of the deployment ID (in case your account has many deployments). A good practice is to always provide the deployment ID whenever you open a support case with us - this speeds-up the overall investigation. > The index has a data:hot allocation requirement on "index.routing.allocation.require.data" from a legacy template on . **cloud-hot-warm-allocation-0** and filtering indices. That's right - this is actually related to [Migrate index allocation filters to node roles](https://www.elastic.co/guide/en/elasticsearch/reference/current/migrate-index-allocation-filters.html#migrate-index-allocation-filters). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 24, 2021, 3:18am UTC](https://discuss.elastic.co/t/resolve-unassigned-shards/276793/7 "2021-07-24T03:18:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.