Lost whole data after restarting ES

Elastic Stack Elasticsearch
1

Hello,

We have 6 indexes on 5 nodes each shard on a node with no replic.

I had a red health on ES today , which appears to be that one shard of one
index unassigned due to full disk space,
After emptying the server from dummy data. I restarted the node that should
have the unassigned node , but it didn't work at All.

Then restarted the whole nodes and had 5 shards unassigned.

I have on my configurations
cluster.routing.allocation.disable_allocation: true

After that we changed the configurations to
cluster.routing.allocation.disable_new_allocation: true
cluster.routing.allocation.disable_allocation: true
cluster.routing.allocation.allow_primary: true

and restarted the whole node.
After that the whole shards become unassigned and we discovered that the
data on the hard disk is also removed.

I want to make sure, what we did wrong so we wont that again.

How can I recover unassigned shard for future ? is it by restart ?

Dina

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c9933972-d0d1-4afc-85c8-d1b5058352da%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

You had disabled replica? Sorry , then I have no answer.

I guess you lost a shard and with the broken shard, the whole index can be
gone. It is a known risk you can lose an index if you run out of disk space
without having replica on an unaffected node.

Shard replica level 1 is active by default for good reason. They are for
failsafety, beside query load balancing.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGB3GtT0a9LLXdsmX%2BWJ7pa%2BavXbUmXPDL0kvj4R_q72A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.