# Cluster discovery worked once, then never again **URL:** \ **Category:** Elasticsearch\ **Created:** [January 15, 2014, 8:51pm UTC](https://discuss.elastic.co/t/cluster-discovery-worked-once-then-never-again/15258 "2014-01-15T20:51:29Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![klarrey](https://avatars.discourse-cdn.com/v4/letter/k/4af34b/32.png) [@klarrey](https://discuss.elastic.co/u/klarrey)\ **Post date:** [January 15, 2014, 8:51pm UTC](https://discuss.elastic.co/t/cluster-discovery-worked-once-then-never-again/15258/1 "2014-01-15T20:51:29Z")
Just getting started with elasticsearch. Once we set it up on two machines, they automatically detected each other, formed a cluster, and began replication (though it didn't complete). Since then, they never connect. So far I haven't been able to find any successful fixes or resources to troubleshoot the issue. The first computer is Windows XP. The second is Windows 7. We also fired up a third (Windows 7) that was unable to connect with either. On the Windows 7 computer, however, we sometimes get the following logs in the elasticsearch.log when the XP starts elastic search: org.elasticsearch.transport.ConnectTransportException: [Zemo, Helmut][inet[/XXX.XXX.XXX.XXX:9300]] connect\_timeout[30s] ``` at ``` org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:711) ``` at ``` org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:640) ``` at ``` org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:608) ``` at ``` org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129) ``` at ``` org.elasticsearch.discovery.zen.ping.multicast.MulticastZenPing$Receiver$2.run(MulticastZenPing.java:546) ``` at ``` java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ``` at ``` java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ``` at java.lang.Thread.run(Thread.java:744) ``` Caused by: java.net.ConnectException: Connection timed out: no further information: /XXX.XXX.XXX.XXX:9300 ``` at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at ``` sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ``` at ``` org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ``` at ``` org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ``` ... 3 more ``` [2014-01-14 18:40:27,842][WARN][discovery.zen.ping.multicast] [Grey, Elaine] failed to connect to requesting node [Zemo, Helmut][ncfQANnkRmODwvV\_E3f-cw][inet[/XXX.XXX.XXX.XXX:9300]] org.elasticsearch.transport.ConnectTransportException: [Zemo, Helmut][inet[/XXX.XXX.XXX.XXX:9300]] connect\_timeout[30s] ``` at ``` org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:711) ``` at ``` org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:640) ``` at ``` org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:608) ``` at ``` org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:129) ``` at ``` org.elasticsearch.discovery.zen.ping.multicast.MulticastZenPing$Receiver$2.run(MulticastZenPing.java:546) ``` at ``` java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ``` at ``` java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ``` at java.lang.Thread.run(Thread.java:744) ``` Caused by: java.net.ConnectException: Connection timed out: no further information: /XXX.XXX.XXX.XXX:9300 ``` at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at ``` sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) ``` at ``` org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ``` at ``` org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ``` at ``` org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) ... 3 more Does anyone know the issue or any resources that walk through how to troubleshoot these issues? Everything I read seems to assume these issues don't exist because ElasticSearch is so easy to get up and running. Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a67796f5-5b6a-4072-bb35-8790e7c7377a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a67796f5-5b6a-4072-bb35-8790e7c7377a%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![brian\_yoder](https://avatars.discourse-cdn.com/v4/letter/b/f1d935/32.png) [@brian\_yoder](https://discuss.elastic.co/u/brian_yoder)\ **Post date:** [January 15, 2014, 9:31pm UTC](https://discuss.elastic.co/t/cluster-discovery-worked-once-then-never-again/15258/2 "2014-01-15T21:31:23Z")
I guess the usual question need to be answered: Are the Java versions exactly the same on all systems? And just to be sure, are the Elasticsearch versions exactly the same on all systems? > _Does anyone know the issue or any resources that walk through how to > troubleshoot these issues? Everything I read seems to assume these issues > don't exist because Elasticsearch is so easy to get up and running._ It is, on Solaris, Linux, and Mac OS X. I have never tried running ES on Windows. But a mix of XP and 7.... Maybe this is a case where you might wish to try unicast discovery to isolate Windows networking issues across old and new versions of the OS? Or, try it on the two Windows 7 systems and leave the XP system out of the picture; can you keep them talking to each other? Just some thoughts... Brian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e7fbaad8-9e40-461a-a327-9fd665573c3a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e7fbaad8-9e40-461a-a327-9fd665573c3a%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![klarrey](https://avatars.discourse-cdn.com/v4/letter/k/4af34b/32.png) [@klarrey](https://discuss.elastic.co/u/klarrey)\ **Post date:** [January 15, 2014, 9:51pm UTC](https://discuss.elastic.co/t/cluster-discovery-worked-once-then-never-again/15258/3 "2014-01-15T21:51:44Z")
Embarrassing - we had set the program exception in Windows Firewall on the Windows 7 computer, but we forgotten that Windows Firewall even existed in XP...which apparently it did, way back then. It was supposed to notify every time it blocked a program but for whatever reason wasn't notifying us, but it seems clear that was the issue as it is working now. Apologies, and many thanks Brian. On Wednesday, January 15, 2014 3:31:23 PM UTC-6, InquiringMind wrote: > I guess the usual question need to be answered: Are the Java versions > exactly the same on all systems? > > And just to be sure, are the Elasticsearch versions exactly the same on > all systems? > > > _Does anyone know the issue or any resources that walk through how to > > troubleshoot these issues? Everything I read seems to assume these issues > > don't exist because Elasticsearch is so easy to get up and running._ > > It is, on Solaris, Linux, and Mac OS X. I have never tried running ES on > Windows. But a mix of XP and 7.... Maybe this is a case where you might > wish to try unicast discovery to isolate Windows networking issues across > old and new versions of the OS? Or, try it on the two Windows 7 systems and > leave the XP system out of the picture; can you keep them talking to each > other? > > Just some thoughts... > > Brian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d16a6042-3fd4-4c9f-a3a1-05b777841f1b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d16a6042-3fd4-4c9f-a3a1-05b777841f1b%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![klarrey](https://avatars.discourse-cdn.com/v4/letter/k/4af34b/32.png) [@klarrey](https://discuss.elastic.co/u/klarrey)\ **Post date:** [January 15, 2014, 11:05pm UTC](https://discuss.elastic.co/t/cluster-discovery-worked-once-then-never-again/15258/4 "2014-01-15T23:05:32Z")
There may still be similar issues though. If anyone has tips on finding out when or where there might be firewall or security issues obstructing communication, please let me know. Currently we still have problems when the XP node closes Elasticsearch (making the Windows 7 node master) and then tries to reconnect. The error logs on the XP show nothing, but the logs on Windows 7 show what is below. However, if the Windows 7 node is shut down making the XP node the master again, everything seems to work fine again. [2014-01-15 16:48:12,778][INFO][cluster.service] [NodeHelios] added {[NodeDevelopment01][UZmP9gC\_Q9es46k6o9OWTw][inet[/XXX.XXX.XXX.XXX:9300]],}, reason: zen-disco-receive(join from node[[NodeDevelopment01][UZmP9gC\_Q9es46k6o9OWTw][inet[/XXX.XXX.XXX.XXX:9300]]]) [2014-01-15 16:48:12,873][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [4] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,876][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [10] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,878][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [6] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,879][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [12] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,887][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [3] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,890][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [2] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,890][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [9] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,891][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [8] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,891][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [5] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,892][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [11] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,893][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [1] and action [cluster/nodeIndexCreated], resetting [2014-01-15 16:48:12,894][WARN][transport.netty] [NodeHelios] Message not fully read (request) for [7] and action [cluster/nodeIndexCreated], resetting On Wednesday, January 15, 2014 3:51:44 PM UTC-6, [kla...@sfile.com](mailto:kla...@sfile.com) wrote: > Embarrassing - we had set the program exception in Windows Firewall on the > Windows 7 computer, but we forgotten that Windows Firewall even existed in > XP...which apparently it did, way back then. It was supposed to notify > every time it blocked a program but for whatever reason wasn't notifying > us, but it seems clear that was the issue as it is working now. Apologies, > and many thanks Brian. > > On Wednesday, January 15, 2014 3:31:23 PM UTC-6, InquiringMind wrote: > > > I guess the usual question need to be answered: Are the Java versions > > exactly the same on all systems? > > > > And just to be sure, are the Elasticsearch versions exactly the same on > > all systems? > > > > > _Does anyone know the issue or any resources that walk through how to > > > troubleshoot these issues? Everything I read seems to assume these issues > > > don't exist because Elasticsearch is so easy to get up and running._ > > > > It is, on Solaris, Linux, and Mac OS X. I have never tried running ES on > > Windows. But a mix of XP and 7.... Maybe this is a case where you might > > wish to try unicast discovery to isolate Windows networking issues across > > old and new versions of the OS? Or, try it on the two Windows 7 systems and > > leave the XP system out of the picture; can you keep them talking to each > > other? > > > > Just some thoughts... > > > > Brian -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c609015a-6cbb-4e1c-a41c-13a76848590d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c609015a-6cbb-4e1c-a41c-13a76848590d%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:56am UTC](https://discuss.elastic.co/t/cluster-discovery-worked-once-then-never-again/15258/5 "2017-07-06T01:56:34Z")