ERROR: no_shard_available_action_exception after restoring the snapshot

Elastic Stack Elasticsearch
,
1

Hello,

I am using ES 8.13 docker and try to restore the snapshot.
I am using _restore ES API.

_snapshot/my_repository/snapshot_1/_restore?wait_for_completion=true
{
  "indices": "test-000001"
}

Result -

{
    "snapshot": {
        "snapshot": "snapshot_1",
        "indices": [
            "test-000001"
        ],
        "shards": {
            "total": 1,
            "failed": 1,
            "successful": 0
        }
    }
}

I checked the docker logs as well.

[T#2]","log.logger":"org.elasticsearch.snapshots.RestoreService","elasticsearch.cluster.uuid":"o9y_nhi1SfyEtlGKmhdLQQ","elasticsearch.node.id":"XtVeRwJLQdGjmqhxQ4nokg","elasticsearch.node.name":"elasticsearch","elasticsearch.cluster.name":"docker-cluster"}
{"@timestamp":"2024-08-16T10:41:36.070Z", "log.level": "INFO",  "current.health":"RED","message":"Cluster health status changed from [YELLOW] to [RED] (reason: [reconcile-desired-balance]).","previous.health":"YELLOW","reason":"reconcile-desired-balance" , "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[elasticsearch-81][masterService#updateTask][T#2]","log.logger":"org.elasticsearch.cluster.routing.allocation.AllocationService","elasticsearch.cluster.uuid":"o9y_nhi1SfyEtlGKmhdLQQ","elasticsearch.node.id":"XtVeRwJLQdGjmqhxQ4nokg","elasticsearch.node.name":"elasticsearch-81","elasticsearch.cluster.name":"docker-cluster"}

These are the indices what we have after restoring:

red    open   test-000001                         xVK8undETUG75c58bxWBDQ   1   1
2

See this troubleshooting guide for further info.

1 Like
3 
/_cat/shards?v=true&h=index,shard,prirep,state,node,unassigned.reason&s=state

Result

index                                                         shard prirep state      node             unassigned.reason
test-000001                                                   0     r      UNASSIGNED                  NEW_INDEX_RESTORED
test-000001                                                   0     p      UNASSIGNED                  NEW_INDEX_RESTORED

_cluster/allocation/explain?filter_path=index,node_allocation_decisions.node_name,node_allocation_decisions.deciders.*

Result

{
    "index": "test-000001",
    "node_allocation_decisions": [
        {
            "node_name": "elasticsearch",
            "deciders": [
                {
                    "decider": "restore_in_progress",
                    "decision": "NO",
                    "explanation": "shard has failed to be restored from the snapshot [my_repository:snapshot_1/04TxOY85QaipfMx3f2e2ag] - manually close or delete the index [test-000001] in order to retry to restore the snapshot again or use the reroute API to force the allocation of an empty primary shard. Details: [restore_source[my_repository/snapshot_1]]"
                },
                {
                    "decider": "disk_threshold",
                    "decision": "NO",
                    "explanation": "the node is above the low watermark cluster setting [cluster.routing.allocation.disk.watermark.low=85%], having less than the minimum required [137gb] free space, actual free: [77.9gb], actual used: [91.4%]"
                }
            ]
        }
    ]
}

I tried with reroute API. but it didn't work.

4

This suggests you don't have enough disk space to restore the snapshot.

1 Like