# Reliability of ec2 discovery? **URL:** \ **Category:** Elasticsearch\ **Created:** [November 8, 2012, 12:11am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615 "2012-11-08T00:11:53Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [November 8, 2012, 12:11am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615/1 "2012-11-08T00:11:53Z")
After starting a second elasticsearch instance on a new machine, I noticed that neither instance could see the other after a while. Documents added to an index via the first instance wouldn't show up in queries to the same index on the second instance. Restarting the second instance got the missing documents to show up, but the instances still couldn't see each other. Only after restarting the first instance did both instances see each other again. No errors or warnings were logged. Has this ever happened to anyone else? - elasticsearch 0.19.11 - elasticsearch-cloud-aws 1.9 - elasticsearch.yml: -- ---
**Author:** ![Chris\_Male](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chris_male/32/2607_2.png) [@Chris\_Male](https://discuss.elastic.co/u/Chris_Male)\ **Post date:** [November 8, 2012, 4:59am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615/2 "2012-11-08T04:59:13Z")
Hi Eric, Is the problem repeatable? If so, are you able to gather the logs at TRACE level so we can see the interaction between the two instances. Are their any other nodes in the cluster? On Thursday, November 8, 2012 11:11:53 AM UTC+11, Eric Jain wrote: > After starting a second elasticsearch instance on a new machine, I noticed > that neither instance could see the other after a while. Documents added to > an index via the first instance wouldn't show up in queries to the same > index on the second instance. Restarting the second instance got the > missing documents to show up, but the instances still couldn't see each > other. Only after restarting the first instance did both instances see each > other again. No errors or warnings were logged. > > Has this ever happened to anyone else? > > - elasticsearch 0.19.11 > > - elasticsearch-cloud-aws 1.9 > > - elasticsearch.yml: -- ---
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [November 8, 2012, 8:11am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615/3 "2012-11-08T08:11:28Z")
On Wed, Nov 7, 2012 at 8:59 PM, Chris Male [gento0nz@gmail.com](mailto:gento0nz@gmail.com) wrote: > Is the problem repeatable? If so, are you able to gather the logs at TRACE > level so we can see the interaction between the two instances. Are their > any other nodes in the cluster? No, it's not reproducible, but I'll need to watch out for this in future (i.e. monitor the number of cluster members). There were just two machines involved. Each machine had an elasticsearch instance, plus an application with a no-data node. -- ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [November 8, 2012, 8:21am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615/4 "2012-11-08T08:21:58Z")
In my experience, EC2 discovery is very reliable. There have been some issues with nodes not finding each other when other region then `us-east-1` was used, make sure to include the region info if it happens again. -- ---
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [November 8, 2012, 8:30am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615/5 "2012-11-08T08:30:58Z")
On Thu, Nov 8, 2012 at 12:21 AM, Karel Minařík [karel.minarik@gmail.com](mailto:karel.minarik@gmail.com) wrote: > In my experience, EC2 discovery is very reliable. There have been some > issues with nodes not finding each other when other region then `us-east-1` > was used, make sure to include the region info if it happens again. I do have region set to 'us-east-1'. -- ---
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [December 12, 2012, 9:49am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615/6 "2012-12-12T09:49:41Z")
Just had another case where a node started on a new machine wouldn't see existing nodes on two other ec2 machines, until I restarted the existing nodes. Can't reproduce... elasticsearch 0.20.1 / cloud-aws 1.10.0 -- ---
**Author:** ![Eric\_Jain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_jain/32/834_2.png) [@Eric\_Jain](https://discuss.elastic.co/u/Eric_Jain)\ **Post date:** [December 12, 2012, 10:13am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615/7 "2012-12-12T10:13:01Z")
On Wed, Dec 12, 2012 at 1:49 AM, Eric Jain [eric.jain@gmail.com](mailto:eric.jain@gmail.com) wrote: > Just had another case where a node started on a new machine wouldn't see > existing nodes on two other ec2 machines, until I restarted the existing > nodes. Can't reproduce... > > elasticsearch 0.20.1 / cloud-aws 1.10.0 2012-12-12 09:23:47,858 [WARN] org.elasticsearch.discovery - [Cyber] waited for 30s and no initial state was set by the discovery -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:00am UTC](https://discuss.elastic.co/t/reliability-of-ec2-discovery/9615/8 "2017-07-06T03:00:09Z")