# ElasticSearch EC2 autodiscovery apparently flaky **URL:** \ **Category:** Elasticsearch\ **Created:** [August 12, 2013, 5:53pm UTC](https://discuss.elastic.co/t/elasticsearch-ec2-autodiscovery-apparently-flaky/13180 "2013-08-12T17:53:19Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![David\_Cross](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@David\_Cross](https://discuss.elastic.co/u/David_Cross)\ **Post date:** [August 12, 2013, 5:53pm UTC](https://discuss.elastic.co/t/elasticsearch-ec2-autodiscovery-apparently-flaky/13180/1 "2013-08-12T17:53:19Z")
I have setup an elasticsearch cluster using cloud-aws and it appears that randomly nodes will not join the cluster; I don't see any error messages, just the message that nothing was received in ~3seconds, and the current node is elected as master. I have the AWS security group is to allow 9200-9400 TCP between all elasticsearch nodes. Inevitably if I just restart the java instance on the node it comes back immediately. Finally, in the cases I have observed this it has been on a newly instantiated instance; is there a possibility there is some startup lag on security groups within AWS? Is there additional information that I could provide that would be useful? Has anyone else seen this before? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [August 12, 2013, 7:13pm UTC](https://discuss.elastic.co/t/elasticsearch-ec2-autodiscovery-apparently-flaky/13180/2 "2013-08-12T19:13:27Z")
I saw that also recently. When starting, ES node takes some seconds to get answer back from EC2 API. After a restart, answer was immediate. I suppose that's the same cause here. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 12 août 2013 à 19:53, David Cross [dcross@whosay.com](mailto:dcross@whosay.com) a écrit : I have setup an elasticsearch cluster using cloud-aws and it appears that randomly nodes will not join the cluster; I don't see any error messages, just the message that nothing was received in ~3seconds, and the current node is elected as master. I have the AWS security group is to allow 9200-9400 TCP between all elasticsearch nodes. Inevitably if I just restart the java instance on the node it comes back immediately. Finally, in the cases I have observed this it has been on a newly instantiated instance; is there a possibility there is some startup lag on security groups within AWS? ## Is there additional information that I could provide that would be useful? Has anyone else seen this before? You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![David\_Cross](https://avatars.discourse-cdn.com/v4/letter/d/958977/32.png) [@David\_Cross](https://discuss.elastic.co/u/David_Cross)\ **Post date:** [August 12, 2013, 8:03pm UTC](https://discuss.elastic.co/t/elasticsearch-ec2-autodiscovery-apparently-flaky/13180/3 "2013-08-12T20:03:53Z")
Ok; In your case did the nodes ever converge? Its one thing (not great, but perhaps livable) if the nodes eventually see each other and start talking after a couple of minutes. But its totally unacceptable if they are forever orphaned. Did you come up with a workaround? (automated?) On Monday, August 12, 2013 3:13:27 PM UTC-4, David Pilato wrote: > I saw that also recently. When starting, ES node takes some seconds to get > answer back from EC2 API. After a restart, answer was immediate. > I suppose that's the same cause here. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 12 août 2013 à 19:53, David Cross \<[dcr...@whosay.com](mailto:dcr...@whosay.com) \\> a > écrit : > > I have setup an elasticsearch cluster using cloud-aws and it appears that > randomly nodes will not join the cluster; I don't see any error messages, > just the message that nothing was received in ~3seconds, and the current > node is elected as master. I have the AWS security group is to allow > 9200-9400 TCP between all elasticsearch nodes. > > Inevitably if I just restart the java instance on the node it comes back > immediately. > > Finally, in the cases I have observed this it has been on a newly > instantiated instance; is there a possibility there is some startup lag on > security groups within AWS? > > Is there additional information that I could provide that would be useful? > Has anyone else seen this before? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [August 12, 2013, 9:20pm UTC](https://discuss.elastic.co/t/elasticsearch-ec2-autodiscovery-apparently-flaky/13180/4 "2013-08-12T21:20:35Z")
I was just running some tests for aws discovery plugin. It was not in production. I just noticed that strange delay I've never seen before. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 12 août 2013 à 22:03, David Cross [dcross@whosay.com](mailto:dcross@whosay.com) a écrit : Ok; In your case did the nodes ever converge? Its one thing (not great, but perhaps livable) if the nodes eventually see each other and start talking after a couple of minutes. But its totally unacceptable if they are forever orphaned. Did you come up with a workaround? (automated?) On Monday, August 12, 2013 3:13:27 PM UTC-4, David Pilato wrote: > I saw that also recently. When starting, ES node takes some seconds to get answer back from EC2 API. After a restart, answer was immediate. > I suppose that's the same cause here. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 12 août 2013 à 19:53, David Cross [dcr...@whosay.com](mailto:dcr...@whosay.com) a écrit : > > I have setup an elasticsearch cluster using cloud-aws and it appears that randomly nodes will not join the cluster; I don't see any error messages, just the message that nothing was received in ~3seconds, and the current node is elected as master. I have the AWS security group is to allow 9200-9400 TCP between all elasticsearch nodes. > > Inevitably if I just restart the java instance on the node it comes back immediately. > > Finally, in the cases I have observed this it has been on a newly instantiated instance; is there a possibility there is some startup lag on security groups within AWS? > > ## Is there additional information that I could provide that would be useful? Has anyone else seen this before? > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:21am UTC](https://discuss.elastic.co/t/elasticsearch-ec2-autodiscovery-apparently-flaky/13180/5 "2017-07-06T02:21:28Z")