Elasticsearch 2.4 node changed after java heap crashed

[motts]

Hello,

As the title suggests, the ES 2.4 instance I had running crashed due to out of java heap error. I increased the memory and restarted ES, but all of the data started showing up as analyzed. Did some more digging and found that /var/lib/elasticsearch/elasticsearch/nodes/0 changed to nodes/1.

This must have affected the mapping. I have found some documentation on how to reindex data, but I cannot find info on how to specify nodes. I am guessing I need to reindex the data from node/1 to node/0 and then delete node/1.

Any advise would be welcomed.

Thanks

[dadoonet (David Pilato)]

That was probably caused by the fact that the nodes was still appearing as "locked".
I don't remember the file name exactly, but if you stop the node (which is currently using dir 1), remove the lock file in dir 0, remove the dir 1 and restart elasticsearch, you should see again all your data.

[motts]

Ok. I will dig around and see what I can find.

[motts]

The nodes are both on the same virtual machine. I am not sure if stopping elasticsearch, removing the entire node 1 directory, and restart elasticsearch will bring back the old data. I have looked through the documents online, and I do not recall seeing anything about this.

I have looked at reindexing the data from node 1 to node 0, but I cannot find any supported documents that say this can be done.

Do you happen to have any more information?

Thanks

[motts]

I found a node.lock file, and did some googling on that. It seems the node.lock file is associated with elastic using that dir to store files. However, in my lab I stopped elastic, moved all the node 0 files to node 1 (including the node.lock file), and restarted elastic, but all the new incoming data was written to the original node 0 and a new node.lock file was placed there. There must be something else associated with elastic using node 1 dir instead of node 0 dir.