# Cached Elasticsearch Information **URL:** \ **Category:** Elasticsearch\ **Created:** [January 14, 2015, 5:22pm UTC](https://discuss.elastic.co/t/cached-elasticsearch-information/21643 "2015-01-14T17:22:14Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Albion\_Baucom](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@Albion\_Baucom](https://discuss.elastic.co/u/Albion_Baucom)\ **Post date:** [January 14, 2015, 5:22pm UTC](https://discuss.elastic.co/t/cached-elasticsearch-information/21643/1 "2015-01-14T17:22:14Z")
I am new to ELK and I am still using a dev environment with real data to understand how the pipeline works. Recently I had nodes networking between them that were part of a 4 node cluster: 1 master node, no data and 3 data-only nodes. These were working fine up to the point that they lost connectivity between themselves. I am using a unicast cluster setup as multicast was not working on my OpenStack cluster (a question for another future post). When I rebooted the nodes and tried to bring the master back up it tried to join the previous master instance. Clearly there is information cached about the previous running instance that needs to be flushed. As this is a dev environment, I copied the config file and blew away the elastic search directory, copied the config back and tried to restart elasticsearch. Curiously it is still trying to join the old master, even though no other processes are running. Obviously this cached data lives somewhere else on the system and I have yet to find it. Perhaps someone can point me in the right direction here. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 15, 2015, 2:41am UTC](https://discuss.elastic.co/t/cached-elasticsearch-information/21643/2 "2015-01-15T02:41:49Z")
It doesn't cache this sort of info, it'll read what is in the config file. Are you using puppet/chef/other for config management perchance? These could be over writing your config. On 15 January 2015 at 06:22, Albion Baucom [albionb@gene.com](mailto:albionb@gene.com) wrote: > I am new to ELK and I am still using a dev environment with real data to > understand how the pipeline works. > > Recently I had nodes networking between them that were part of a 4 node > cluster: 1 master node, no data and 3 data-only nodes. These were working > fine up to the point that they lost connectivity between themselves. I am > using a unicast cluster setup as multicast was not working on my OpenStack > cluster (a question for another future post). > > When I rebooted the nodes and tried to bring the master back up it tried > to join the previous master instance. Clearly there is information cached > about the previous running instance that needs to be flushed. As this is a > dev environment, I copied the config file and blew away the Elasticsearch > directory, copied the config back and tried to restart elasticsearch. > Curiously it is still trying to join the old master, even though no other > processes are running. Obviously this cached data lives somewhere else on > the system and I have yet to find it. > > Perhaps someone can point me in the right direction here. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X96sBGgJiEqfWc6X9GTHHvfUryCOSpSUS%3DqFj8\_HMcdtA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X96sBGgJiEqfWc6X9GTHHvfUryCOSpSUS%3DqFj8_HMcdtA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Albion\_Baucom](https://avatars.discourse-cdn.com/v4/letter/a/838e76/32.png) [@Albion\_Baucom](https://discuss.elastic.co/u/Albion_Baucom)\ **Post date:** [January 16, 2015, 8:15pm UTC](https://discuss.elastic.co/t/cached-elasticsearch-information/21643/3 "2015-01-16T20:15:22Z")
No configuration management in place. I manually untarred the files. So all configuration information should be contained in the application directory? Here is what I see which is, even after a reboot, a Nightwatch named Elasticsearch instance looking for the phantom "Plantman" master, even though this node is configured to be a master only: [root@es-master elasticsearch-1.4.2]# [2015-01-16 11:57:43,500][INFO][node] [_Nightwatch_] version[1.4.2], pid[24139], build[927caff/2014-12-16T14:11:12Z] [2015-01-16 11:57:43,501][INFO][node] [Nightwatch] initializing ... [2015-01-16 11:57:43,505][INFO][plugins] [Nightwatch] loaded , sites [2015-01-16 11:57:46,701][INFO][node] [Nightwatch] initialized [2015-01-16 11:57:46,701][INFO][node] [Nightwatch] starting ... [2015-01-16 11:57:46,903][INFO][transport] [Nightwatch] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish\_address {inet[/10.0.0.45:9300]} [2015-01-16 11:57:46,919][INFO][discovery] [Nightwatch] lsflogs/A6CISFuzRkK31rL2dFFdNA [2015-01-16 11:57:50,187][INFO][discovery.zen] [Nightwatch] _failed to send join request to master [[Plantman]_[VOmbX4yORHiObk7Q3D7tbQ][es-master][inet[/10.0.0.45:9300]]{data=false, master=true}], reason [RemoteTransportException[[Nightwatch][inet[/10.0.0.45:9300]][internal:discovery/zen/join]]; nested: ElasticsearchIllegalStateException[Node [[Nightwatch][A6CISFuzRkK31rL2dFFdNA][es-master][inet[/10.0.0.45:9300]]{data=false, master=true}] not master for join request from [[Nightwatch][A6CISFuzRkK31rL2dFFdNA][es-master][inet[/10.0.0.45:9300]]{data=false, master=true}]]; ], tried [3] times My workaround was to rename the cluster. Perhaps there was another node with that information and it was confusing the master? But you answered my question which is there is no information written outside of the application directory that could cache settings for future clusters if I tear them down and rebuild them. Thanks Albion On Wednesday, January 14, 2015 at 6:42:23 PM UTC-8, Mark Walkom wrote: > It doesn't cache this sort of info, it'll read what is in the config file. > > Are you using puppet/chef/other for config management perchance? These > could be over writing your config. > > On 15 January 2015 at 06:22, Albion Baucom \<[alb...@gene.com](mailto:alb...@gene.com) \\> > wrote: > > > I am new to ELK and I am still using a dev environment with real data to > > understand how the pipeline works. > > > > Recently I had nodes networking between them that were part of a 4 node > > cluster: 1 master node, no data and 3 data-only nodes. These were working > > fine up to the point that they lost connectivity between themselves. I am > > using a unicast cluster setup as multicast was not working on my OpenStack > > cluster (a question for another future post). > > > > When I rebooted the nodes and tried to bring the master back up it tried > > to join the previous master instance. Clearly there is information cached > > about the previous running instance that needs to be flushed. As this is a > > dev environment, I copied the config file and blew away the Elasticsearch > > directory, copied the config back and tried to restart elasticsearch. > > Curiously it is still trying to join the old master, even though no other > > processes are running. Obviously this cached data lives somewhere else on > > the system and I have yet to find it. > > > > Perhaps someone can point me in the right direction here. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/12876abb-4c7c-4ef8-934f-d0afac8b8c3b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/12876abb-4c7c-4ef8-934f-d0afac8b8c3b%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 16, 2015, 11:24pm UTC](https://discuss.elastic.co/t/cached-elasticsearch-information/21643/4 "2015-01-16T23:24:05Z")
If you are just untarring and starting the ES binary, it'll use the defaults, which is multicast and default cluster name. So it'll do a search for any other nodes and if they have the same cluster name and can reach each other on the network, they will try to form a cluster, that is what you are seeing. On 17 January 2015 at 09:15, Albion Baucom [albionb@gene.com](mailto:albionb@gene.com) wrote: > No configuration management in place. I manually untarred the files. So > all configuration information should be contained in the application > directory? > > Here is what I see which is, even after a reboot, a Nightwatch named > Elasticsearch instance looking for the phantom "Plantman" master, even > though this node is configured to be a master only: > > [root@es-master elasticsearch-1.4.2]# > [2015-01-16 11:57:43,500][INFO][node] [_Nightwatch_] > version[1.4.2], pid[24139], build[927caff/2014-12-16T14:11:12Z] > [2015-01-16 11:57:43,501][INFO][node] [Nightwatch] > initializing ... > [2015-01-16 11:57:43,505][INFO][plugins] [Nightwatch] > loaded , sites > [2015-01-16 11:57:46,701][INFO][node] [Nightwatch] > initialized > [2015-01-16 11:57:46,701][INFO][node] [Nightwatch] > starting ... > [2015-01-16 11:57:46,903][INFO][transport] [Nightwatch] > bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish\_address {inet[/ > 10.0.0.45:9300]} > [2015-01-16 11:57:46,919][INFO][discovery] [Nightwatch] > lsflogs/A6CISFuzRkK31rL2dFFdNA > [2015-01-16 11:57:50,187][INFO][discovery.zen] [Nightwatch] _failed > to send join request to master [[Plantman]_ > [VOmbX4yORHiObk7Q3D7tbQ][es-master][inet[/10.0.0.45:9300]]{data=false, > master=true}], reason > [RemoteTransportException[[Nightwatch][inet[/10.0.0.45:9300]][internal:discovery/zen/join]]; > nested: ElasticsearchIllegalStateException[Node > [[Nightwatch][A6CISFuzRkK31rL2dFFdNA][es-master][inet[/10.0.0.45:9300]]{data=false, > master=true}] not master for join request from > [[Nightwatch][A6CISFuzRkK31rL2dFFdNA][es-master][inet[/10.0.0.45:9300]]{data=false, > master=true}]]; ], tried [3] times > > My workaround was to rename the cluster. Perhaps there was another node > with that information and it was confusing the master? > > But you answered my question which is there is no information written > outside of the application directory that could cache settings for future > clusters if I tear them down and rebuild them. > > Thanks > Albion > > On Wednesday, January 14, 2015 at 6:42:23 PM UTC-8, Mark Walkom wrote: > > > It doesn't cache this sort of info, it'll read what is in the config file. > > > > Are you using puppet/chef/other for config management perchance? These > > could be over writing your config. > > > > On 15 January 2015 at 06:22, Albion Baucom [alb...@gene.com](mailto:alb...@gene.com) wrote: > > > > > I am new to ELK and I am still using a dev environment with real data to > > > understand how the pipeline works. > > > > > > Recently I had nodes networking between them that were part of a 4 node > > > cluster: 1 master node, no data and 3 data-only nodes. These were working > > > fine up to the point that they lost connectivity between themselves. I am > > > using a unicast cluster setup as multicast was not working on my OpenStack > > > cluster (a question for another future post). > > > > > > When I rebooted the nodes and tried to bring the master back up it tried > > > to join the previous master instance. Clearly there is information cached > > > about the previous running instance that needs to be flushed. As this is a > > > dev environment, I copied the config file and blew away the Elasticsearch > > > directory, copied the config back and tried to restart elasticsearch. > > > Curiously it is still trying to join the old master, even though no other > > > processes are running. Obviously this cached data lives somewhere else on > > > the system and I have yet to find it. > > > > > > Perhaps someone can point me in the right direction here. > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/90187018-dde9-447d-b211-d806fd46cec8%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/12876abb-4c7c-4ef8-934f-d0afac8b8c3b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/12876abb-4c7c-4ef8-934f-d0afac8b8c3b%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/12876abb-4c7c-4ef8-934f-d0afac8b8c3b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/12876abb-4c7c-4ef8-934f-d0afac8b8c3b%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-7Hn2ka9ExL27YE5Z6X96L0\_jN7pDpA6U0P%2BO%2BM860fA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-7Hn2ka9ExL27YE5Z6X96L0_jN7pDpA6U0P%2BO%2BM860fA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:38am UTC](https://discuss.elastic.co/t/cached-elasticsearch-information/21643/5 "2017-07-06T00:38:16Z")