I deleted index like files into OS when I had a diskspace problem and now have to fix it

Elastic Stack Elasticsearch
1

Hi guys
I have a problem with Elasticsearch cluster
I deleted index like files into OS, when I had a diskspace problem
I deleted index through this command find ./data/nodes/0/indices -type d -mtime +30 -exec rm -rf {} \;

What I have:
One node elastic cluster

And now have this problem:

  1. When I sent curl -XGET "localhost:9200/_cat/shards" | grep "UNASSIGNED" | wc -l
    UNASSIGNED idexes - 1780
    STARTED indexes - 143

  2. When I sent curl --user elastic:elastic -XGET localhost:9200/_cat/health?v
    have a response:
    {"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/_cat/health?v]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/_cat/health?v]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}},"status":401}

3)I created new user bin/Elasticsearch-users useradd newadmin -p PASSWORD -r superuser
for restoring elaatic password
But when I sent request for changing password have this message:
Request:

curl -s --user [new user with superuser role] -XPUT "http://localhost:9200/_xpack/security/user/elastic/_password?pretty" -H 'Content-Type: application/json' -d ' 
{ 
"password" : "PASSWORD" 
}'

Answer:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "unavailable_shards_exception",
        "reason" : "[.security-7][0] [1] shardIt, [0] active : Timeout waiting for [1m], request: indices:data/write/update"
      }
    ],
    "type" : "unavailable_shards_exception",
    "reason" : "[.security-7][0] [1] shardIt, [0] active : Timeout waiting for [1m], request: indices:data/write/update"
  },
  "status" : 503
}

Request for showing cluster health:
curl -XGET --user [new user with superuser role] 'localhost:9200/_cat/health?v'

epoch      timestamp cluster          status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1642410739 09:12:19  rule-engine-prod red             1         1    143 143    0    0     1780             0                  -                  7.4%

Request for showing allocation:
curl -XGET --user [new user with superuser role] 'localhost:9200/_cat/allocation?v'

shards disk.indices disk.used disk.avail disk.total disk.percent host        ip          node
   143       63.8gb      93gb    299.7gb    392.7gb           23 10.44.0.125 10.44.0.125 rule-engine-whg3
  1780                                                                                   UNASSIGNED

Please help me, How can I fix it?

2

You have likely lost the data, which is why you should ALWAYS use the APIs to delete indices.

The only way to recover would be to restore a backup, or reindex the data.

2 Likes
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.