# Shards remain UNASSIGNED after \_restore operation

**URL:** <https://discuss.elastic.co/t/shards-remain-unassigned-after--restore-operation/75239>\
**Category:** Elasticsearch\
**Created:** [February 15, 2017, 6:53pm UTC](https://discuss.elastic.co/t/shards-remain-unassigned-after--restore-operation/75239 "2017-02-15T18:53:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jspooner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jspooner/32/12984_2.png) [@jspooner](https://discuss.elastic.co/u/jspooner)\
**Post date:** [February 15, 2017, 6:53pm UTC](https://discuss.elastic.co/t/shards-remain-unassigned-after--restore-operation/75239/1 "2017-02-15T18:53:35Z")

</div>

When running this **[\_restore](https://www.elastic.co/guide/en/elasticsearch/reference/5.2/modules-snapshots.html)** command not all of my indexes are restored and this command never returns.

```
curl -s -XPOST 'localhost:9200/_snapshot/s3_repository/2017-02-10-17/_restore?wait_for_completion=true-d' '{
  "ignore_unavailable": true,
  "include_global_state": false
}'

```

You can run **[\_cluster/allocation/explain](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-allocation-explain.html)** API to see why things are screwed up.

Q1: What the heck is `"last_allocation_status" : "no_attempt"`?  
Q1.1: **"cannot allocate because allocation is not permitted to any of the nodes"** Why is allocation not permitted on any nodes? Other indexes restored with no issues.

```
curl -s -XGET "localhost:9200/_cluster/allocation/explain?pretty"
{
  "index" : "sightings-geohex-2016-01-04",
  "shard" : 0,
  "primary" : false,
  "current_state" : "unassigned",
  "unassigned_info" : {
    "reason" : "NEW_INDEX_RESTORED",
    "at" : "2017-02-15T18:21:43.960Z",
    "details" : "restore_source[s3_repository/2017-02-10]",
    "last_allocation_status" : "no_attempt"
  },
  "can_allocate" : "no",
  "allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",
  "node_allocation_decisions" : [
    {
      "node_id" : "1wXza4AESD-SO4S4kWHQNA",
      "node_name" : "1wXza4A",
      "transport_address" : "10.148.14.172:9300",
      "node_decision" : "no",
      "deciders" : [
        {
          "decider" : "replica_after_primary_active",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        },
        {
          "decider" : "throttling",
          "decision" : "NO",
          "explanation" : "primary shard for this replica is not yet active"
        }
      ]
    }
      ]
    }
  ]
}

```

Let's look at the primary node specifically

curl -XGET "$INSTANCE\_IP:9200/\_cluster/allocation/explain?pretty" -d '{  
"index": "sightings-geohex-2016-01-01",  
"shard": 0,  
"primary": true  
}'

Q: Why does the backup look for this **\_id**?  
Q: Is the **\_id** different than the **[node.name](http://node.name)**?

```
{
    "allocate_explanation": "cannot allocate because allocation is not permitted to any of the nodes",
    "can_allocate": "no",
    "current_state": "unassigned",
    "index": "sightings-geohex-2016-01-01",
    "node_allocation_decisions": [
        {
            "deciders": [
                {
                    "decider": "filter",
                    "decision": "NO",
                    "explanation": "initial allocation of the index is only allowed on nodes [_id:\"HUbsbDLGRrWwoQtKlXP3Vw\"]"
                }
            ],
            "node_decision": "no",
            "node_id": "2LS9wxZuS72Wt_lbUybEIw",
            "node_name": "2LS9wxZ",
            "transport_address": "10.91.139.169:9300",
            "weight_ranking": 1
        },
        {
            "deciders": [
                {
                    "decider": "filter",
                    "decision": "NO",
                    "explanation": "initial allocation of the index is only allowed on nodes [_id:\"HUbsbDLGRrWwoQtKlXP3Vw\"]"
                }
            ],
            "node_decision": "no",
            "node_id": "ZRduycOURwyyGn1SZrd72Q",
            "node_name": "ZRduycO",
            "transport_address": "10.61.190.175:9300",
            "weight_ranking": 2
        },
        {
            "deciders": [
                {
                    "decider": "filter",
                    "decision": "NO",
                    "explanation": "initial allocation of the index is only allowed on nodes [_id:\"HUbsbDLGRrWwoQtKlXP3Vw\"]"
                }
            ],
            "node_decision": "no",
            "node_id": "cbs3JIE5T4e3b1kr_3wCAg",
            "node_name": "cbs3JIE",
            "transport_address": "10.164.223.27:9300",
            "weight_ranking": 3
        },
        {
            "deciders": [
                {
                    "decider": "filter",
                    "decision": "NO",
                    "explanation": "initial allocation of the index is only allowed on nodes [_id:\"HUbsbDLGRrWwoQtKlXP3Vw\"]"
                }
            ],
            "node_decision": "no",
            "node_id": "LKi-FBxBRcOqrnsZs_n4Fw",
            "node_name": "HUbsbDLGRrWwoQtKlXP3Vw",
            "transport_address": "10.170.35.169:9300",
            "weight_ranking": 4
        }
    ],
    "primary": true,
    "shard": 0,
    "unassigned_info": {
        "at": "2017-02-15T18:21:43.960Z",
        "details": "restore_source[s3_repository/2017-02-10-17:36:49]",
        "last_allocation_status": "no",
        "reason": "NEW_INDEX_RESTORED"
    }
}

```

Now taking a loot at the **Index Metadata**.

Q2: What is `allocation.initial_recovery`? I can't find it in any docs. Is this looking for a node with that name?  
Q3: This index was created as a result of a **[\_shrink](https://www.elastic.co/guide/en/elasticsearch/reference/5.2/indices-shrink-index.html)** operation. The shrink source index was deleted before taking the **\_snapshot**. Routing was set on the **source index** to `"index.routing.allocation.require._name": "shrink_node_name"`. However the new index automatically moved to other data nodes before the **\_snapshot** was taken.

```
{
	"state": "open",
	"settings": {
		"index": {
			"routing": {
				"allocation": {
					"initial_recovery": {
						"_id": "HUbsbDLGRrWwoQtKlXP3Vw"
					}
				}
			},
			"allocation": {
				"max_retries": "1"
			},
			"number_of_shards": "1",
			"shrink": {
				"source": {
					"name": "bulk-sightings-geohex-2016-01-04",
					"uuid": "Y3tQt30zRTKUiQvG39VtyA"
				}
			},
			"provided_name": "sightings-geohex-2016-01-04",
			"creation_date": "1486754454536",
			"number_of_replicas": "1",
			"uuid": "TDvEhpYeTmulcHYgSa0coQ",
			"version": {
				"created": "5020099"
			}
		}
	},
	"mappings": {
		"sighting": {
			"properties": {
				SNIP
			}
		}
	},
	"aliases": [],
	"primary_terms": {
		"0": 1
	},
	"in_sync_allocations": {
		"0": [
			"1HL22qjlTxyi1IllOqACVQ",
			"EZ__d7liTaaTeO2Z6BlJUA"
		]
	}
}

```

Q4: How do I get the **[\_restore](https://www.elastic.co/guide/en/elasticsearch/reference/5.2/modules-snapshots.html#_restore)** operation to finish correctly?

---

<div class="post-metadata">

**Author:** ![jspooner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jspooner/32/12984_2.png) [@jspooner](https://discuss.elastic.co/u/jspooner)\
**Post date:** [February 16, 2017, 12:25am UTC](https://discuss.elastic.co/t/shards-remain-unassigned-after--restore-operation/75239/2 "2017-02-16T00:25:19Z")

</div>

After a few hours of debugging I narrowed the issue down to an issue with \_id. I moved the issue over to a new question

> [@Restoring to a different cluster with version 5.2.0](https://discuss.elastic.co/t/restoring-to-a-different-cluster/75255):
>
> In the guides for [restoring to a different cluster](https://www.elastic.co/guide/en/elasticsearch/reference/5.2/modules-snapshots.html#_restoring_to_a_different_cluster) it says If indices in the original cluster were assigned to particular nodes using shard allocation filtering, the same rules will be enforced in the new cluster. Therefore if the new cluster doesn’t contain nodes with appropriate attributes that a restored index can be allocated on, such index will not be successfully restored unless these index allocation settings are changed during restore operation. In my situation I have an index that w…

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 16, 2017, 12:26am UTC](https://discuss.elastic.co/t/shards-remain-unassigned-after--restore-operation/75239/3 "2017-03-16T00:26:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
