# Regular Split Brain Issue Running on 2 VMs in Windows Azure (Unicast Discovery) **URL:** \ **Category:** Elasticsearch\ **Created:** [May 6, 2013, 2:36pm UTC](https://discuss.elastic.co/t/regular-split-brain-issue-running-on-2-vms-in-windows-azure-unicast-discovery/11837 "2013-05-06T14:36:58Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Nariman\_Haghighi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nariman_haghighi/32/1065_2.png) [@Nariman\_Haghighi](https://discuss.elastic.co/u/Nariman_Haghighi)\ **Post date:** [May 6, 2013, 2:36pm UTC](https://discuss.elastic.co/t/regular-split-brain-issue-running-on-2-vms-in-windows-azure-unicast-discovery/11837/1 "2013-05-06T14:36:58Z")
We're running ES on 2 Azure VMs using unicast discovery and opening ports 9200 and 9300 on both nodes. The initial start is always successful but we've started to notice split brain pattern emerging a few hours into each deployment. The configuration is as follows: {"path":{"data":"F:\","work":"F:\"},"cluster":{"name":"FiveAces.Coffee.Web"},"node":{"name":"FiveAces.Coffee.Web\_IN\_0"},"discovery":{"zen":{"ping":{"multicast":{"enabled":false},"unicast":{"hosts":["10.241.238.26","10.241.182.18"]}}}}} And this is the pattern that's happening almost regularly (which subsequently leads to data loss after each side is restarted): _NODE1:_ [2013-05-05 22:00:47,123][INFO][node] [FiveAces.Coffee.Web\_IN\_1] {0.90.0}[2164]: initializing ... [2013-05-05 22:00:47,524][INFO][plugins] [FiveAces.Coffee.Web\_IN\_1] loaded [], sites [head] [2013-05-05 22:00:53,259][INFO][node] [FiveAces.Coffee.Web\_IN\_1] {0.90.0}[2164]: initialized [2013-05-05 22:00:53,264][INFO][node] [FiveAces.Coffee.Web\_IN\_1] {0.90.0}[2164]: starting ... [2013-05-05 22:00:53,506][INFO][transport] [FiveAces.Coffee.Web\_IN\_1] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish\_address {inet[/10.241.182.18:9300]} [2013-05-05 22:01:00,800][INFO][discovery.zen] [FiveAces.Coffee.Web\_IN\_1] master\_left [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]], reason [do not exists on master, act as master failure] [2013-05-05 22:01:00,831][INFO][cluster.service] [FiveAces.Coffee.Web\_IN\_1] detected\_master [FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]], added {[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]],}, reason: zen-disco-receive(from master [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]]) [2013-05-05 22:01:00,911][INFO][discovery] [FiveAces.Coffee.Web\_IN\_1] FiveAces.Coffee.Web/lH-lp\_jsQ4WhwwFJO3B-kA [2013-05-05 22:01:00,912][INFO][cluster.service] [FiveAces.Coffee.Web\_IN\_1] master {new [FiveAces.Coffee.Web\_IN\_1][lH-lp\_jsQ4WhwwFJO3B-kA][inet[/10.241.182.18:9300]], previous [FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]}, removed {[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]],}, reason: zen-disco-master\_failed ([FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]) [2013-05-05 22:01:02,352][INFO][http] [FiveAces.Coffee.Web\_IN\_1] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish\_address {inet[/10.241.182.18:9200]} [2013-05-05 22:01:02,354][INFO][node] [FiveAces.Coffee.Web\_IN\_1] {0.90.0}[2164]: started [2013-05-05 22:01:03,912][WARN][discovery.zen] [FiveAces.Coffee.Web\_IN\_1] received cluster state from [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] which is also master but with an older cluster\_state, telling [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] to rejoin the cluster [2013-05-05 22:01:03,965][WARN][discovery.zen] [FiveAces.Coffee.Web\_IN\_1] received cluster state from [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] which is also master but with an older cluster\_state, telling [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] to rejoin the cluster [2013-05-05 22:01:03,966][WARN][discovery.zen] [FiveAces.Coffee.Web\_IN\_1] received cluster state from [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] which is also master but with an older cluster\_state, telling [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] to rejoin the cluster [2013-05-05 22:01:03,966][WARN][discovery.zen] [FiveAces.Coffee.Web\_IN\_1] failed to send rejoin request to [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] org.elasticsearch.transport.SendRequestTransportException: [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]][discovery/zen/rejoin] at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:199) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:171) at org.elasticsearch.discovery.zen.ZenDiscovery$7.execute(ZenDiscovery.java:527) at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:229) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:95) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: org.elasticsearch.transport.NodeNotConnectedException: [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]] Node not connected at org.elasticsearch.transport.netty.NettyTransport.nodeChannel(NettyTransport.java:788) at org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:522) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:184) ... 7 more [2013-05-05 22:01:04,072][WARN][discovery.zen] [FiveAces.Coffee.Web\_IN\_1] failed to send rejoin request to [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] org.elasticsearch.transport.SendRequestTransportException: [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]][discovery/zen/rejoin] at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:199) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:171) at org.elasticsearch.discovery.zen.ZenDiscovery$7.execute(ZenDiscovery.java:527) at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:229) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:95) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: org.elasticsearch.transport.NodeNotConnectedException: [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]] Node not connected at org.elasticsearch.transport.netty.NettyTransport.nodeChannel(NettyTransport.java:788) at org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:522) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:184) ... 7 more [2013-05-05 22:01:03,966][WARN][discovery.zen] [FiveAces.Coffee.Web\_IN\_1] failed to send rejoin request to [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] org.elasticsearch.transport.SendRequestTransportException: [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]][discovery/zen/rejoin] at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:199) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:171) at org.elasticsearch.discovery.zen.ZenDiscovery$7.execute(ZenDiscovery.java:527) at org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:229) at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:95) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: org.elasticsearch.transport.NodeNotConnectedException: [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]] Node not connected at org.elasticsearch.transport.netty.NettyTransport.nodeChannel(NettyTransport.java:788) at org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:522) at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:184) ... 7 more _NODE2:_ [2013-05-05 22:00:48,930][INFO][node] [FiveAces.Coffee.Web\_IN\_0] {0.90.0}[2640]: initializing ... [2013-05-05 22:00:49,133][INFO][plugins] [FiveAces.Coffee.Web\_IN\_0] loaded [], sites [head] [2013-05-05 22:00:54,125][INFO][node] [FiveAces.Coffee.Web\_IN\_0] {0.90.0}[2640]: initialized [2013-05-05 22:00:54,127][INFO][node] [FiveAces.Coffee.Web\_IN\_0] {0.90.0}[2640]: starting ... [2013-05-05 22:00:54,310][INFO][transport] [FiveAces.Coffee.Web\_IN\_0] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish\_address {inet[/10.241.238.26:9300]} [2013-05-05 22:00:57,389][INFO][cluster.service] [FiveAces.Coffee.Web\_IN\_0] new\_master [FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]], reason: zen-disco-join (elected\_as\_master) [2013-05-05 22:00:57,406][INFO][discovery] [FiveAces.Coffee.Web\_IN\_0] FiveAces.Coffee.Web/CVJT6uiFR4OEAzzXyRL\_yQ [2013-05-05 22:00:58,618][INFO][http] [FiveAces.Coffee.Web\_IN\_0] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish\_address {inet[/10.241.238.26:9200]} [2013-05-05 22:00:58,619][INFO][node] [FiveAces.Coffee.Web\_IN\_0] {0.90.0}[2640]: started [2013-05-05 22:01:00,532][INFO][gateway] [FiveAces.Coffee.Web\_IN\_0] recovered [1] indices into cluster\_state [2013-05-05 22:01:00,794][INFO][cluster.service] [FiveAces.Coffee.Web\_IN\_0] added {[FiveAces.Coffee.Web\_IN\_1][lH-lp\_jsQ4WhwwFJO3B-kA][inet[/10.241.182.18:9300]],}, reason: zen-disco-receive(join from node[[FiveAces.Coffee.Web\_IN\_1][lH-lp\_jsQ4WhwwFJO3B-kA][inet[/10.241.182.18:9300]]]) Could this be tied to the transport module using a bind range of 9300-9400 by default? Wondering if I should give it a narrow range of one or two ports that I can then open up between the two VMs. The data loss that results from this is unacceptable, any suggestions on how to avoid the split brain scenario (other than moving to 3 nodes) would be appreciated. Regards, N. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![imdhmd](https://avatars.discourse-cdn.com/v4/letter/i/d07c76/32.png) [@imdhmd](https://discuss.elastic.co/u/imdhmd)\ **Post date:** [May 7, 2013, 4:52am UTC](https://discuss.elastic.co/t/regular-split-brain-issue-running-on-2-vms-in-windows-azure-unicast-discovery/11837/2 "2013-05-07T04:52:41Z")
We use elasticsearch-zookeeper plugin to address the split brain issue: [GitHub - sonian/elasticsearch-zookeeper](https://github.com/sonian/elasticsearch-zookeeper) This helps avoiding split-brain because, the master election process is now externalised to zookeeper service. On Monday, May 6, 2013 8:06:58 PM UTC+5:30, Nariman Haghighi wrote: > We're running ES on 2 Azure VMs using unicast discovery and opening ports > 9200 and 9300 on both nodes. > > The initial start is always successful but we've started to notice split > brain pattern emerging a few hours into each deployment. > > The configuration is as follows: > > {"path":{"data":"F:\","work":"F:\"},"cluster":{"name":"FiveAces.Coffee.Web"},"node":{"name":"FiveAces.Coffee.Web\_IN\_0"},"discovery":{"zen":{"ping":{"multicast":{"enabled":false},"unicast":{"hosts":["10.241.238.26","10.241.182.18"]}}}}} > > And this is the pattern that's happening almost regularly (which > subsequently leads to data loss after each side is restarted): > > _NODE1:_ > > [2013-05-05 22:00:47,123][INFO][node] > [FiveAces.Coffee.Web\_IN\_1] {0.90.0}[2164]: initializing ... > [2013-05-05 22:00:47,524][INFO][plugins] > [FiveAces.Coffee.Web\_IN\_1] loaded , sites [head] > [2013-05-05 22:00:53,259][INFO][node] > [FiveAces.Coffee.Web\_IN\_1] {0.90.0}[2164]: initialized > [2013-05-05 22:00:53,264][INFO][node] > [FiveAces.Coffee.Web\_IN\_1] {0.90.0}[2164]: starting ... > [2013-05-05 22:00:53,506][INFO][transport] > [FiveAces.Coffee.Web\_IN\_1] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, > publish\_address {inet[/10.241.182.18:9300]} > [2013-05-05 22:01:00,800][INFO][discovery.zen] > [FiveAces.Coffee.Web\_IN\_1] master\_left > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]], > reason [do not exists on master, act as master failure] > [2013-05-05 22:01:00,831][INFO][cluster.service] > [FiveAces.Coffee.Web\_IN\_1] detected\_master > [FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]], > added > {[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]],}, > reason: zen-disco-receive(from master > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]]) > [2013-05-05 22:01:00,911][INFO][discovery] > [FiveAces.Coffee.Web\_IN\_1] FiveAces.Coffee.Web/lH-lp\_jsQ4WhwwFJO3B-kA > [2013-05-05 22:01:00,912][INFO][cluster.service] > [FiveAces.Coffee.Web\_IN\_1] master {new > [FiveAces.Coffee.Web\_IN\_1][lH-lp\_jsQ4WhwwFJO3B-kA][inet[/10.241.182.18:9300]], > previous > [FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]}, > removed > {[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]],}, > reason: zen-disco-master\_failed > ([FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]) > [2013-05-05 22:01:02,352][INFO][http] > [FiveAces.Coffee.Web\_IN\_1] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]}, > publish\_address {inet[/10.241.182.18:9200]} > [2013-05-05 22:01:02,354][INFO][node] > [FiveAces.Coffee.Web\_IN\_1] {0.90.0}[2164]: started > [2013-05-05 22:01:03,912][WARN][discovery.zen] > [FiveAces.Coffee.Web\_IN\_1] received cluster state from > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > which is also master but with an older cluster\_state, telling > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > to rejoin the cluster > [2013-05-05 22:01:03,965][WARN][discovery.zen] > [FiveAces.Coffee.Web\_IN\_1] received cluster state from > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > which is also master but with an older cluster\_state, telling > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > to rejoin the cluster > [2013-05-05 22:01:03,966][WARN][discovery.zen] > [FiveAces.Coffee.Web\_IN\_1] received cluster state from > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > which is also master but with an older cluster\_state, telling > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > to rejoin the cluster > [2013-05-05 22:01:03,966][WARN][discovery.zen] > [FiveAces.Coffee.Web\_IN\_1] failed to send rejoin request to > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > org.elasticsearch.transport.SendRequestTransportException: > [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]][discovery/zen/rejoin] > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:199) > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:171) > at > org.elasticsearch.discovery.zen.ZenDiscovery$7.execute(ZenDiscovery.java:527) > at > org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:229) > at > org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:95) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > Caused by: org.elasticsearch.transport.NodeNotConnectedException: > [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]] Node not connected > at > org.elasticsearch.transport.netty.NettyTransport.nodeChannel(NettyTransport.java:788) > at > org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:522) > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:184) > ... 7 more > [2013-05-05 22:01:04,072][WARN][discovery.zen] > [FiveAces.Coffee.Web\_IN\_1] failed to send rejoin request to > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > org.elasticsearch.transport.SendRequestTransportException: > [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]][discovery/zen/rejoin] > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:199) > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:171) > at > org.elasticsearch.discovery.zen.ZenDiscovery$7.execute(ZenDiscovery.java:527) > at > org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:229) > at > org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:95) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > Caused by: org.elasticsearch.transport.NodeNotConnectedException: > [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]] Node not connected > at > org.elasticsearch.transport.netty.NettyTransport.nodeChannel(NettyTransport.java:788) > at > org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:522) > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:184) > ... 7 more > [2013-05-05 22:01:03,966][WARN][discovery.zen] > [FiveAces.Coffee.Web\_IN\_1] failed to send rejoin request to > [[FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]]] > org.elasticsearch.transport.SendRequestTransportException: > [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]][discovery/zen/rejoin] > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:199) > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:171) > at > org.elasticsearch.discovery.zen.ZenDiscovery$7.execute(ZenDiscovery.java:527) > at > org.elasticsearch.cluster.service.InternalClusterService$2.run(InternalClusterService.java:229) > at > org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:95) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > Caused by: org.elasticsearch.transport.NodeNotConnectedException: > [FiveAces.Coffee.Web\_IN\_0][inet[/10.241.238.26:9300]] Node not connected > at > org.elasticsearch.transport.netty.NettyTransport.nodeChannel(NettyTransport.java:788) > at > org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:522) > at > org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:184) > ... 7 more > > _NODE2:_ > > [2013-05-05 22:00:48,930][INFO][node] > [FiveAces.Coffee.Web\_IN\_0] {0.90.0}[2640]: initializing ... > [2013-05-05 22:00:49,133][INFO][plugins] > [FiveAces.Coffee.Web\_IN\_0] loaded , sites [head] > [2013-05-05 22:00:54,125][INFO][node] > [FiveAces.Coffee.Web\_IN\_0] {0.90.0}[2640]: initialized > [2013-05-05 22:00:54,127][INFO][node] > [FiveAces.Coffee.Web\_IN\_0] {0.90.0}[2640]: starting ... > [2013-05-05 22:00:54,310][INFO][transport] > [FiveAces.Coffee.Web\_IN\_0] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, > publish\_address {inet[/10.241.238.26:9300]} > [2013-05-05 22:00:57,389][INFO][cluster.service] > [FiveAces.Coffee.Web\_IN\_0] new\_master > [FiveAces.Coffee.Web\_IN\_0][CVJT6uiFR4OEAzzXyRL\_yQ][inet[/10.241.238.26:9300]], > reason: zen-disco-join (elected\_as\_master) > [2013-05-05 22:00:57,406][INFO][discovery] > [FiveAces.Coffee.Web\_IN\_0] FiveAces.Coffee.Web/CVJT6uiFR4OEAzzXyRL\_yQ > [2013-05-05 22:00:58,618][INFO][http] > [FiveAces.Coffee.Web\_IN\_0] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]}, > publish\_address {inet[/10.241.238.26:9200]} > [2013-05-05 22:00:58,619][INFO][node] > [FiveAces.Coffee.Web\_IN\_0] {0.90.0}[2640]: started > [2013-05-05 22:01:00,532][INFO][gateway] > [FiveAces.Coffee.Web\_IN\_0] recovered [1] indices into cluster\_state > [2013-05-05 22:01:00,794][INFO][cluster.service] > [FiveAces.Coffee.Web\_IN\_0] added > {[FiveAces.Coffee.Web\_IN\_1][lH-lp\_jsQ4WhwwFJO3B-kA][inet[/10.241.182.18:9300]],}, > reason: zen-disco-receive(join from > node[[FiveAces.Coffee.Web\_IN\_1][lH-lp\_jsQ4WhwwFJO3B-kA][inet[/10.241.182.18:9300]]]) > > Could this be tied to the transport module using a bind range of 9300-9400 > by default? Wondering if I should give it a narrow range of one or two > ports that I can then open up between the two VMs. > > The data loss that results from this is unacceptable, any suggestions on > how to avoid the split brain scenario (other than moving to 3 nodes) would > be appreciated. > > Regards, > N. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:37am UTC](https://discuss.elastic.co/t/regular-split-brain-issue-running-on-2-vms-in-windows-azure-unicast-discovery/11837/3 "2017-07-06T02:37:59Z")