Dangling index problem

Elastic Stack Elasticsearch
1

hi all,
please provide me some solution asap
i have some indexes on local files at my server.after shutdown and restart
it detected some other server as master which had no index in it.i realized
the problem and after shutting down the scarp node i restarted my original
node but all my indices become dangled and i cant find any way to get them
back.
please please give me a solution cuz those indices i were using in
production.

Thanking you all in advance
Ashutosh raikvar

2

I'm afraid you can not get back your documents.
As far as understand your problem:
You have one node with x docs.
You have node two running with 0 docs but with the same clustername.

You stop node1 and start it again.

So your node1 recognize your node2 which is a master node and told to node1 that
there is 0 docs in the index.
So, node1 delete all remaining index/docs.

One thing you can try is to stop node1. Then to stop node2.
Then start node1 and see if you get back your docs.

Try to see the size of your node1 data dir to see if documents remains here.
If your dir is almost empty, you won't recover your docs.

My 2 cents.
David.

Le 27 juillet 2012 à 15:26, ashu raikvar ashu.raikvar@gmail.com a écrit :

hi all,
please provide me some solution asap
i have some indexes on local files at my server.after shutdown and restart it
detected some other server as master which had no index in it.i realized the
problem and after shutting down the scarp node i restarted my original node
but all my indices become dangled and i cant find any way to get them back.
please please give me a solution cuz those indices i were using in
production.

Thanking you all in advance
Ashutosh raikvar

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

3

Which version elasticsearch were you running?

On Friday, July 27, 2012 10:36:47 AM UTC-4, David Pilato wrote:

I'm afraid you can not get back your documents.

As far as understand your problem:

You have one node with x docs.

You have node two running with 0 docs but with the same clustername.

You stop node1 and start it again.

So your node1 recognize your node2 which is a master node and told to
node1 that there is 0 docs in the index.

So, node1 delete all remaining index/docs.

One thing you can try is to stop node1. Then to stop node2.

Then start node1 and see if you get back your docs.

Try to see the size of your node1 data dir to see if documents remains
here.

If your dir is almost empty, you won't recover your docs.

My 2 cents.

David.

Le 27 juillet 2012 à 15:26, ashu raikvar ashu.raikvar@gmail.com a
écrit :

hi all,
please provide me some solution asap
i have some indexes on local files at my server.after shutdown and restart
it detected some other server as master which had no index in it.i realized
the problem and after shutting down the scarp node i restarted my original
node but all my indices become dangled and i cant find any way to get them
back.
please please give me a solution cuz those indices i were using in
production.

Thanking you all in advance
Ashutosh raikvar

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

4

elasticsearch 0.19.7

On Fri, Jul 27, 2012 at 9:31 PM, Igor Motov imotov@gmail.com wrote:

Which version elasticsearch were you running?

On Friday, July 27, 2012 10:36:47 AM UTC-4, David Pilato wrote:

I'm afraid you can not get back your documents.

As far as understand your problem:

You have one node with x docs.

You have node two running with 0 docs but with the same clustername.

You stop node1 and start it again.

So your node1 recognize your node2 which is a master node and told to
node1 that there is 0 docs in the index.

So, node1 delete all remaining index/docs.

One thing you can try is to stop node1. Then to stop node2.

Then start node1 and see if you get back your docs.

Try to see the size of your node1 data dir to see if documents remains
here.

If your dir is almost empty, you won't recover your docs.

My 2 cents.

David.

Le 27 juillet 2012 à 15:26, ashu raikvar ashu.raikvar@gmail.com a
écrit :

hi all,
please provide me some solution asap
i have some indexes on local files at my server.after shutdown and
restart it detected some other server as master which had no index in it.i
realized the problem and after shutting down the scarp node i restarted my
original node but all my indices become dangled and i cant find any way to
get them back.
please please give me a solution cuz those indices i were using in
production.

Thanking you all in advance
Ashutosh raikvar

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

5

I have all my indices file intact in my local file system [38gb size] but
my production node does not recognize indices and throws dangling index
problem.

On Friday, July 27, 2012 6:56:26 PM UTC+5:30, ashu raikvar wrote:

hi all,
please provide me some solution asap
i have some indexes on local files at my server.after shutdown and restart
it detected some other server as master which had no index in it.i realized
the problem and after shutting down the scarp node i restarted my original
node but all my indices become dangled and i cant find any way to get them
back.
please please give me a solution cuz those indices i were using in
production.

Thanking you all in advance
Ashutosh raikvar

6

So, first, backup theses files.
Then, is the dirname under data is the same as your cluster name ?
What happens if you only start your local node?
Can you see your index and documents?

--

Le 28 juil. 2012 à 07:27, ashu raikvar ashu.raikvar@gmail.com a écrit :

I have all my indices file intact in my local file system [38gb size] but my production node does not recognize indices and throws dangling index problem.

On Friday, July 27, 2012 6:56:26 PM UTC+5:30, ashu raikvar wrote:
hi all,
please provide me some solution asap
i have some indexes on local files at my server.after shutdown and restart it detected some other server as master which had no index in it.i realized the problem and after shutting down the scarp node i restarted my original node but all my indices become dangled and i cant find any way to get them back.
please please give me a solution cuz those indices i were using in production.

Thanking you all in advance
Ashutosh raikvar

7

no i cant see my indices all of them become dangling as soon as i start
production node.i did recovery from backup i made of whole cluster which
saved me from disaster production node recognized the indices from the rar
backup.but can some one explain me the fundamentals of _state files in
cluster and the shards?

On Friday, July 27, 2012 6:56:26 PM UTC+5:30, ashu raikvar wrote:

hi all,
please provide me some solution asap
i have some indexes on local files at my server.after shutdown and restart
it detected some other server as master which had no index in it.i realized
the problem and after shutting down the scarp node i restarted my original
node but all my indices become dangled and i cant find any way to get them
back.
please please give me a solution cuz those indices i were using in
production.

Thanking you all in advance
Ashutosh raikvar