# Elasticsearch: 2-node cluster with failover **URL:** \ **Category:** Elasticsearch\ **Created:** [May 28, 2015, 8:40am UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703 "2015-05-28T08:40:00Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Prakhar\_Mishra](https://avatars.discourse-cdn.com/v4/letter/p/f19dbf/32.png) [@Prakhar\_Mishra](https://discuss.elastic.co/u/Prakhar_Mishra)\ **Post date:** [May 28, 2015, 8:40am UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703/1 "2015-05-28T08:40:00Z")
I am using **Elasticsearch 1.5.2** and trying to setup a 2-node cluster. These 2 nodes are primarily for failover strategy (if any one node goes down, the other one is still there to handle requests), I don't need to divide primary shards or something like that, (total data is no more than 500mb on hard-disk). Everything goes well, until **Split Brains** thing kicks in. Now, since I don't have much data, I don't feel any requirement of 3 nodes. And I want to have failover mechanism too. Which means, _discovery.zen.minimum\_master\_nodes_ cannot be more than _1_. Now, I have two questions: 1. Is there any configuration possible, which could overcome **2 master nodes** or **Split Brains** problem? 2. If not, what all other options do I have to make it work? Like, keeping both in different clusters (one online, other one offline) and updating offline with online, time to time, for the time when online cluster goes down. Or, do I have to go for 3-node cluster? I am going on production environment. Please help. Env Info: Two machines (VMWare driven), having 4 cores, 8gb RAM and CentOS. We are firing 7-8 queries (1 _\_search_ and other _\_count_) per search. Indexing op won't be very frequent (until we sync existing indices from DB through _\_bulk_ API), but searching is going to be quite frequent. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 28, 2015, 9:04am UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703/2 "2015-05-28T09:04:55Z")
Add another node (master only : data false) with minimal HEAP and set minimum master nodes to 2 or set minimum master nodes to 2 but in that case your cluster will stop working in case of failure of any node or network failure -- David Pilato - Developer | Evangelist > **[Elastic — The Search AI Company](https://www.elastic.co/)** > > Power insights and outcomes with The Elastic Search AI Platform. See into your data and find answers that matter with enterprise solutions designed to help you accelerate time to insight. Try Elastic ... @dadoonet [https://twitter.com/dadoonet](https://twitter.com/dadoonet) | @elasticsearchfr [https://twitter.com/elasticsearchfr](https://twitter.com/elasticsearchfr) | @scrutmydocs [https://twitter.com/scrutmydocs](https://twitter.com/scrutmydocs) > Le 28 mai 2015 à 10:40, prakhar [prakhar.mishra1990@gmail.com](mailto:prakhar.mishra1990@gmail.com) a écrit : > > I am using _Elasticsearch 1.5.2_ and trying to setup a 2-node cluster. These > 2 nodes are primarily for failover strategy (if any one node goes down, the > other one is still there to handle requests), I don't need to divide primary > shards or something like that, (total data is no more than 500mb on > hard-disk). > > Everything goes well, until _Split Brains_ thing kicks in. Now, since I > don't have much data, I don't feel any requirement of 3 nodes. And I want to > have failover mechanism too. Which means, > /discovery.zen.minimum\_master\_nodes/ cannot be more than /1/. > > Now, I have two questions: > > 1. Is there any configuration possible, which could overcome _2 master > nodes_ or \*Split Brains \*problem? > 2. If not, what all other options do I have to make it work? Like, keeping > both in different clusters (one online, other one offline) and updating > offline with online, time to time, for the time when online cluster goes > down. Or, do I have to go for 3-node cluster? > > I am going on production environment. Please help. > > Env Info: Two machines (VMWare driven), having 4 cores, 8gb RAM and CentOS. > We are firing 7-8 queries (1 /\_search/ and other /\_count/) per search. > Indexing op won't be very frequent (until we sync existing indices from DB > through /\_bulk/ API), but searching is going to be quite frequent. > > -- > View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949.html](http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1432802400465-4074949.post%40n3.nabble.com](https://groups.google.com/d/msgid/elasticsearch/1432802400465-4074949.post%40n3.nabble.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/42DC41A9-8F90-4BB9-BBBC-35D57105A809%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/42DC41A9-8F90-4BB9-BBBC-35D57105A809%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Prakhar\_Mishra](https://avatars.discourse-cdn.com/v4/letter/p/f19dbf/32.png) [@Prakhar\_Mishra](https://discuss.elastic.co/u/Prakhar_Mishra)\ **Post date:** [May 28, 2015, 10:25am UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703/3 "2015-05-28T10:25:31Z")
Thanks for your reply. If I add one **master only** node on one of the two existing servers (2 nodes on the same physical server), with **HEAP** , lets say, _100mb_; will that work? Or I have to add another machine for that? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 28, 2015, 11:05am UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703/4 "2015-05-28T11:05:16Z")
That's not enough heap, you want 1GB minimum. PS - We're moving to [https://discuss.elastic.co/](https://discuss.elastic.co/), please join us there for any future discussions! On 28 May 2015 at 20:25, prakhar [prakhar.mishra1990@gmail.com](mailto:prakhar.mishra1990@gmail.com) wrote: > Thanks for your reply. > > If I add one _master only_ node on one of the two existing servers (2 nodes > on the same physical server), with _HEAP_, lets say, /100mb/; will that > work? Or I have to add another machine for that? > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949p4074961.html](http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949p4074961.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1432808731045-4074961.post%40n3.nabble.com](https://groups.google.com/d/msgid/elasticsearch/1432808731045-4074961.post%40n3.nabble.com) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8ZLP5A1k80t-D61CKLZNZ%3D8PdP4w8tsA7\_O4eBWdYq4A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8ZLP5A1k80t-D61CKLZNZ%3D8PdP4w8tsA7_O4eBWdYq4A%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 28, 2015, 11:08am UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703/5 "2015-05-28T11:08:07Z")
On Thursday, May 28, 2015 at 12:25 CEST, prakhar [prakhar.mishra1990@gmail.com](mailto:prakhar.mishra1990@gmail.com) wrote: > If I add one _master only_ node on one of the two existing servers (2 > nodes on the same physical server), with _HEAP_, lets say, /100mb/; > will that work? Or I have to add another machine for that? That should work, but it obviously means that if the machine with two Elasticsearch nodes is taken out your cluster will be inoperable (but it will survive if the one-node machine dies). -- Magnus Bäck | Software Engineer, Development Tools [magnus.back@sonymobile.com](mailto:magnus.back@sonymobile.com) | Sony Mobile Communications ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/20150528110806.GA25289%40seldlx20533.corpusers.net](https://groups.google.com/d/msgid/elasticsearch/20150528110806.GA25289%40seldlx20533.corpusers.net). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Prakhar\_Mishra](https://avatars.discourse-cdn.com/v4/letter/p/f19dbf/32.png) [@Prakhar\_Mishra](https://discuss.elastic.co/u/Prakhar_Mishra)\ **Post date:** [May 28, 2015, 1:11pm UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703/6 "2015-05-28T13:11:33Z")
Sorry to bother you @Mark, but this term **appropriate heap size** always haunts me. Suppose, I have 2 virtual machines (4 core, 8 gb RAM each). According to elasticsearch's documentation, half of RAM should be allocated to elasticsearch. So, this is what I came up with: Node1: Elasticsearch 1.5.2 **Master-Data** node with 4 gb HEAP ( **server1** ) Node2: Elasticsearch 1.5.2 **Master-Only** (no data) node with 1 gb HEAP ( **server1** ) Node3: Elasticsearch 1.5.2 **Master-Data** node with 4 gb HEAP ( **server2** ) Do you think this is a sane strategy? Moreover, if I make **Master-Only** node (Node2) invisible to load balancer i.e. I simply won't include it at all, will it still require 1 gb HEAP? Please forgive me if I am asking something too naive. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 28, 2015, 10:05pm UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703/7 "2015-05-28T22:05:54Z")
Master only nodes don't need to leverage FS caching as they hold no data. So you can easily increase their heap to 75% of system. However you really don't want to have less than 1GB of heap irrespective of the role it plays. PS - We're moving to [https://discuss.elastic.co/](https://discuss.elastic.co/), please join us there for any future discussions! On 28 May 2015 at 23:11, prakhar [prakhar.mishra1990@gmail.com](mailto:prakhar.mishra1990@gmail.com) wrote: > Sorry to bother you @Mark, but this term _appropriate heap size_ always > haunts me. Suppose, I have 2 virtual machines (4 core, 8 gb RAM each). > According to elasticsearch's documentation, half of RAM should be allocated > to elasticsearch. So, this is what I came up with: > > Node1: Elasticsearch 1.5.2 _Master-Data_ node with 4 gb HEAP (_server1_) > Node2: Elasticsearch 1.5.2 _Master-Only_(no data) node with 1 gb HEAP > (_server1_) > Node3: Elasticsearch 1.5.2 _Master-Data_ node with 4 gb HEAP (_server2_) > > Do you think this is a sane strategy? Moreover, if I make _Master-Only_ > node > (Node2) invisible to load balancer i.e. I simply won't include it at all, > will it still require 1 gb HEAP? > Please forgive me if I am asking something too naive. > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949p4074968.html](http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949p4074968.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1432818693390-4074968.post%40n3.nabble.com](https://groups.google.com/d/msgid/elasticsearch/1432818693390-4074968.post%40n3.nabble.com) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_Oj\_d3qG%3DQoTW0A\_ChjdXigoj2Rn2iOQurnGpCcU21dQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_Oj_d3qG%3DQoTW0A_ChjdXigoj2Rn2iOQurnGpCcU21dQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:11am UTC](https://discuss.elastic.co/t/elasticsearch-2-node-cluster-with-failover/23703/8 "2017-07-06T00:11:08Z")