ERROR: no_shard_available_action_exception

Elastic Stack Elasticsearch
1

Hi,

Does someone know how to fix this issue "no_shard_available_action_exception". Encountered this error after I restore my cluster, index was restored but the documents was not restored.

Kindly see the error below:

'
{
"error" : {
"root_cause" : [
{
"type" : "no_shard_available_action_exception",
"reason" : null,
"index_uuid" : "XGNHFe4eS-uxmtk9VbiLRA",
"shard" : "0",
"index" : "employees"
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "employees",
"node" : null,
"reason" : {
"type" : "no_shard_available_action_exception",
"reason" : null,
"index_uuid" : "XGNHFe4eS-uxmtk9VbiLRA",
"shard" : "0",
"index" : "employees"
}
}
]
},
"status" : 503
}

2

How did you restore it? How did you know the index was restored?

Elasticsearch doesn't seem to be able to see it, so it'd be good to understand what steps you took and what you can see.

3

These are the steps that I executed:

  1. REGISTER A REPOSITORY:

curl -X PUT "localhost:9200/_snapshot/backup_01?pretty" -H 'Content-Type: application/json' -d'
{
"type": "fs",
"settings": {
"location": "/mnt/backup"
}
}
'

  1. CREATE A SNAPSHOT:

curl -X PUT "localhost:9200/_snapshot/backup_01/snapshot_01?wait_for_completion=true&pretty"

  1. DELETE ALL INDEX: (To simulate data loss)

curl -XDELETE "localhost:9200/_all"

  1. RESTORE A SNAPSHOT:

curl -X POST "localhost:9200/_snapshot/backup_01/snapshot_01/_restore?pretty" -H 'Content-Type: application/json' -d'
{
"indices": "-.ds-.logs-deprecation.elasticsearch-default-2022.10.19-000001,-.ds-ilm-history-5-2022.10.19-000001, *"
}
'

1 Like
4

Thanks!

What was the response for that restore? Can you share the output from GET _cat/indices?v?

5

Hi @warkolm,

Here is the output,

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
red open employees XGNHFe4eS-uxmtk9VbiLRA 1 1

6

Thanks. It looks like your restore did not work?
I would do a;

curl -X POST "localhost:9200/_snapshot/backup_01/snapshot_01/_restore?pretty" -H 'Content-Type: application/json' -d'
{
"indices": "employees"
}
'

And see what happens. Make sure you capture the response this time :slight_smile:

7

I deleted the employees index and tried to redo the restore using the curl command that you've given. Kindly see the response below:

{
"accepted" : true
}

After the restore I checked the indices again, however the documents in the employees index was not restored. Kindly see the output of GET _cat/indices?v ? below:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
red open employees D_aZSE0zRNeI78dil7tOiA 1 1

8

It'd be really great if you could share the restore request and the output.

9

Here is the restore request and the output:

RESTORE REQUEST:

curl -X POST "localhost:9200/_snapshot/backup_01/snapshot_01/_restore?pretty" -H 'Content-Type: application/json' -d'
{
"indices": "employees"
}
'

OUTPUT:

{
"accepted" : true
}

10

Thanks. Are you sure there was documents in the snapshot? What does the output from this show;

GET _snapshot/backup_01/snapshot_01/_status
1 Like
11

Here is the output:

{
   "snapshots":[
      {
         "snapshot":"snapshot_01",
         "repository":"backup_01",
         "uuid":"UT5wbIXiTp-HpcEwDGF-bA",
         "state":"SUCCESS",
         "include_global_state":true,
         "shards_stats":{
            "initializing":0,
            "started":0,
            "finalizing":0,
            "done":3,
            "failed":0,
            "total":3
         },
         "stats":{
            "incremental":{
               "file_count":27,
               "size_in_bytes":69427
            },
            "total":{
               "file_count":27,
               "size_in_bytes":69427
            },
            "start_time_in_millis":1666156245588,
            "time_in_millis":643
         },
         "indices":{
            ".ds-.logs-deprecation.elasticsearch-default-2022.10.19-000001":{
               "shards_stats":{
                  "initializing":0,
                  "started":0,
                  "finalizing":0,
                  "done":1,
                  "failed":0,
                  "total":1
               },
               "stats":{
                  "incremental":{
                     "file_count":13,
                     "size_in_bytes":41254
                  },
                  "total":{
                     "file_count":13,
                     "size_in_bytes":41254
                  },
                  "start_time_in_millis":1666156245789,
                  "time_in_millis":442
               },
               "shards":{
                  "0":{
                     "stage":"DONE",
                     "stats":{
                        "incremental":{
                           "file_count":13,
                           "size_in_bytes":41254
                        },
                        "total":{
                           "file_count":13,
                           "size_in_bytes":41254
                        },
                        "start_time_in_millis":1666156245789,
                        "time_in_millis":442
                     }
                  }
               }
            },
            ".ds-ilm-history-5-2022.10.19-000001":{
               "shards_stats":{
                  "initializing":0,
                  "started":0,
                  "finalizing":0,
                  "done":1,
                  "failed":0,
                  "total":1
               },
               "stats":{
                  "incremental":{
                     "file_count":7,
                     "size_in_bytes":19258
                  },
                  "total":{
                     "file_count":7,
                     "size_in_bytes":19258
                  },
                  "start_time_in_millis":1666156245789,
                  "time_in_millis":240
               },
               "shards":{
                  "0":{
                     "stage":"DONE",
                     "stats":{
                        "incremental":{
                           "file_count":7,
                           "size_in_bytes":19258
                        },
                        "total":{
                           "file_count":7,
                           "size_in_bytes":19258
                        },
                        "start_time_in_millis":1666156245789,
                        "time_in_millis":240
                     }
                  }
               }
            },
            "employees":{
               "shards_stats":{
                  "initializing":0,
                  "started":0,
                  "finalizing":0,
                  "done":1,
                  "failed":0,
                  "total":1
               },
               "stats":{
                  "incremental":{
                     "file_count":7,
                     "size_in_bytes":8915
                  },
                  "total":{
                     "file_count":7,
                     "size_in_bytes":8915
                  },
                  "start_time_in_millis":1666156245789,
                  "time_in_millis":240
               },
               "shards":{
                  "0":{
                     "stage":"DONE",
                     "stats":{
                        "incremental":{
                           "file_count":7,
                           "size_in_bytes":8915
                        },
                        "total":{
                           "file_count":7,
                           "size_in_bytes":8915
                        },
                        "start_time_in_millis":1666156245789,
                        "time_in_millis":240
                     }
                  }
               }
            }
         }
      }
   ]
}[
12

It doesn't look like your original index had any data in it from what I can see, that appears to be an empty index that was snapshotted.

(Also please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you :slight_smile: )

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.