After starting a second elasticsearch instance on a new machine, I noticed
that neither instance could see the other after a while. Documents added to
an index via the first instance wouldn't show up in queries to the same
index on the second instance. Restarting the second instance got the
missing documents to show up, but the instances still couldn't see each
other. Only after restarting the first instance did both instances see each
other again. No errors or warnings were logged.
Is the problem repeatable? If so, are you able to gather the logs at TRACE
level so we can see the interaction between the two instances. Are their
any other nodes in the cluster?
On Thursday, November 8, 2012 11:11:53 AM UTC+11, Eric Jain wrote:
After starting a second elasticsearch instance on a new machine, I noticed
that neither instance could see the other after a while. Documents added to
an index via the first instance wouldn't show up in queries to the same
index on the second instance. Restarting the second instance got the
missing documents to show up, but the instances still couldn't see each
other. Only after restarting the first instance did both instances see each
other again. No errors or warnings were logged.
Is the problem repeatable? If so, are you able to gather the logs at TRACE
level so we can see the interaction between the two instances. Are their
any other nodes in the cluster?
No, it's not reproducible, but I'll need to watch out for this in
future (i.e. monitor the number of cluster members).
There were just two machines involved. Each machine had an
elasticsearch instance, plus an application with a no-data node.
In my experience, EC2 discovery is very reliable. There have been some
issues with nodes not finding each other when other region then us-east-1
was used, make sure to include the region info if it happens again.
In my experience, EC2 discovery is very reliable. There have been some
issues with nodes not finding each other when other region then us-east-1
was used, make sure to include the region info if it happens again.
Just had another case where a node started on a new machine wouldn't see
existing nodes on two other ec2 machines, until I restarted the existing
nodes. Can't reproduce...
Just had another case where a node started on a new machine wouldn't see
existing nodes on two other ec2 machines, until I restarted the existing
nodes. Can't reproduce...
elasticsearch 0.20.1 / cloud-aws 1.10.0
2012-12-12 09:23:47,858 [WARN] org.elasticsearch.discovery - [Cyber]
waited for 30s and no initial state was set by the discovery
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.