# Deleting Dangling nodes after upgrade **URL:** \ **Category:** Elasticsearch\ **Created:** [March 13, 2012, 10:48pm UTC](https://discuss.elastic.co/t/deleting-dangling-nodes-after-upgrade/6986 "2012-03-13T22:48:21Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Onilton\_Maciel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/onilton_maciel/32/2500_2.png) [@Onilton\_Maciel](https://discuss.elastic.co/u/Onilton_Maciel)\ **Post date:** [March 13, 2012, 10:48pm UTC](https://discuss.elastic.co/t/deleting-dangling-nodes-after-upgrade/6986/1 "2012-03-13T22:48:21Z")
When I upgraded from 0.17.6 to 0.19.0, after I restarted my cluster as 0.19 I get in ES log that there are dangling indices and they were marked for removal after 2 minutes but all my nodes/indices had data before. ES is claiming there is no metadata for them. As I didn't set anything about gateway based on this[http://www.elasticsearch.org/guide/reference/modules/gateway/](http://www.elasticsearch.org/guide/reference/modules/gateway/) [1] I think i am using the local[http://www.elasticsearch.org/guide/reference/modules/gateway/local.html](http://www.elasticsearch.org/guide/reference/modules/gateway/local.html) [2] gateway by default. I know that when you start a empty cluster with a empty node, you needed to set recover\_after\_nodes to assure a node with data comes before cluster start But as my cluster and nodes all already had data. What could be happening? Do I need to start ES from the last Master node? Or something like? Or am I accidentaly deleting the file for local gateway data in the update? If this file exists where is it? [1] [http://www.elasticsearch.org/guide/reference/modules/gateway/](http://www.elasticsearch.org/guide/reference/modules/gateway/) I think i am using the local gateway by default [2] [http://www.elasticsearch.org/guide/reference/modules/gateway/local.html](http://www.elasticsearch.org/guide/reference/modules/gateway/local.html) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [March 14, 2012, 12:22pm UTC](https://discuss.elastic.co/t/deleting-dangling-nodes-after-upgrade/6986/2 "2012-03-14T12:22:20Z")
The dangling indices message means that the index itself (its metadata) does not exists in the recovered global cluster metadata, but there seem to be evidence for them on disk. Can you try and restore the backup files created for the metadata, and start it back with 0.17.x, and then try and upgrade to 0.18 and then to 0.19. The upgrade path from 0.17 to 0.19 was not tested…. On Wednesday, March 14, 2012 at 12:48 AM, Onilton Maciel wrote: > When I upgraded from 0.17.6 to 0.19.0, after I restarted my cluster as 0.19 > I get in ES log that there are dangling indices and they were marked for removal after 2 minutes but all my nodes/indices had data before. ES is claiming there is no metadata for them. > > As I didn't set anything about gateway based on this ([Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/gateway/)) [1] I think i am using the local ([Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/gateway/local.html)) [2] gateway by default. > > I know that when you start a empty cluster with a empty node, you needed to set recover\_after\_nodes to assure a node with data comes before cluster start > > But as my cluster and nodes all already had data. What could be happening? > > Do I need to start ES from the last Master node? Or something like? > > Or am I accidentaly deleting the file for local gateway data in the update? If this file exists where is it? > > [1] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/gateway/) I think i am using the local gateway by default > [2] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/gateway/local.html) ---
**Author:** ![Onilton\_Maciel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/onilton_maciel/32/2500_2.png) [@Onilton\_Maciel](https://discuss.elastic.co/u/Onilton_Maciel)\ **Post date:** [March 14, 2012, 2:37pm UTC](https://discuss.elastic.co/t/deleting-dangling-nodes-after-upgrade/6986/3 "2012-03-14T14:37:53Z")
I don't know where is "backup files created for the metadata" you said. Can you tell me where it is? Also one thing I would like to know is: If I upgrade from a direct version 0.17 -\> 0.18 without using any gateway (so it will be local by default), using the same nodes, same cluster, would be any risk for me to lose my index or get dangling nodes? PS: Assume I turned off my index program and flushed all the indexes before turning off ES. On Wed, Mar 14, 2012 at 8:22 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > The dangling indices message means that the index itself (its metadata) > does not exists in the recovered global cluster metadata, but there seem to > be evidence for them on disk. Can you try and restore the backup files > created for the metadata, and start it back with 0.17.x, and then try and > upgrade to 0.18 and then to 0.19. The upgrade path from 0.17 to 0.19 was > not tested…. > > On Wednesday, March 14, 2012 at 12:48 AM, Onilton Maciel wrote: > > When I upgraded from 0.17.6 to 0.19.0, after I restarted my cluster as 0.19 > I get in ES log that there are dangling indices and they were marked for > removal after 2 minutes but all my nodes/indices had data before. ES is > claiming there is no metadata for them. > > As I didn't set anything about gateway based on this[http://www.elasticsearch.org/guide/reference/modules/gateway/](http://www.elasticsearch.org/guide/reference/modules/gateway/) [1] > I think i am using the local[http://www.elasticsearch.org/guide/reference/modules/gateway/local.html](http://www.elasticsearch.org/guide/reference/modules/gateway/local.html) [2] > gateway by default. > > I know that when you start a empty cluster with a empty node, you needed > to set recover\_after\_nodes to assure a node with data comes before cluster > start > > But as my cluster and nodes all already had data. What could be happening? > > Do I need to start ES from the last Master node? Or something like? > > Or am I accidentaly deleting the file for local gateway data in the > update? If this file exists where is it? > > [1] [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/gateway/) I think > i am using the local gateway by default > [2] > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/gateway/local.html) ---
**Author:** ![Paul\_Smith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_smith/32/1323_2.png) [@Paul\_Smith](https://discuss.elastic.co/u/Paul_Smith)\ **Post date:** [March 14, 2012, 8:08pm UTC](https://discuss.elastic.co/t/deleting-dangling-nodes-after-upgrade/6986/4 "2012-03-14T20:08:55Z")
On 15 March 2012 01:37, Onilton Maciel [oniltonmaciel@gmail.com](mailto:oniltonmaciel@gmail.com) wrote: > I don't know where is "backup files created for the metadata" you said. > Can you tell me where it is? > > Also one thing I would like to know is: > If I upgrade from a direct version 0.17 -\> 0.18 without using any gateway > (so it will be local by default), using the same nodes, same cluster, would > be any risk for me to lose my index or get dangling nodes? > > PS: Assume I turned off my index program and flushed all the indexes > before turning off ES. > > On Wed, Mar 14, 2012 at 8:22 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > > The dangling indices message means that the index itself (its metadata) > > does not exists in the recovered global cluster metadata, but there seem to > > be evidence for them on disk. Can you try and restore the backup files > > created for the I've been meaning to post more details about this myself, but we've also been running 0.17.9 in prod for a while, and planned to jump to 0.19, but found we _must_ upgrade to 0.18.x, and then to 0.19 for the existing indexes to restore correctly. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:36am UTC](https://discuss.elastic.co/t/deleting-dangling-nodes-after-upgrade/6986/5 "2017-07-06T03:36:06Z")
Closed on July 6, 2017, 3:36am UTC