# Cluster setup **URL:** \ **Category:** Elasticsearch\ **Created:** [January 16, 2014, 4:50pm UTC](https://discuss.elastic.co/t/cluster-setup/15275 "2014-01-16T16:50:38Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Tula](https://avatars.discourse-cdn.com/v4/letter/t/838e76/32.png) [@Tula](https://discuss.elastic.co/u/Tula)\ **Post date:** [January 16, 2014, 4:50pm UTC](https://discuss.elastic.co/t/cluster-setup/15275/1 "2014-01-16T16:50:38Z")
Hi, I have 3 ubuntu VM's on a private network, each has 64GB ram. I started ES Beta2 (need it to use term vector feature) on each node with 30GB heap space and with the following changes in the configuration file: discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["host1","host2","host3"] bootstrap.mlockall: true gateway.type: local gateway.recover\_after\_nodes: 2 gateway.expected\_nodes: 3 discovery.zen.minimum\_master\_nodes: 2 Sometimes es processes get killed by OOM killer and when I restart a node I ran into situation like > host1,host2,host3 thinking all three nodes are connected and form a > cluster, and the host3 thinks it is all by itself with status red (names > are random) > host1 and host2 form one cluster and host2 and host3 forms another > cluster. Any idea about what I am doing wrong and any suggestions? I will need to index 20 million documents and have 3 separate indexes with 5 shards 1 replica. Thanks, T -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/62130b83-40aa-468e-9449-0046497d39ff%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/62130b83-40aa-468e-9449-0046497d39ff%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [January 20, 2014, 9:12am UTC](https://discuss.elastic.co/t/cluster-setup/15275/2 "2014-01-20T09:12:05Z")
Hey, I am not a huge fan of the OOM killer to be honest. However, something is going against your plans when the OOM killer kicks in You configured 30GB heap, but you are running out of memory (then most of the time the process which takes the most memory is killed, obviously elasticsearch). But why are you running out of memory? Do you have any other service running on that machine, which eats up system memory? Please check (or disable the OOM killer, but you should find out why it kicks in). Also, use the nodes info API to find out if bootstrap.mlockall setting is really configured correctly on your nodes. --Alex On Thu, Jan 16, 2014 at 5:50 PM, Tula [tulay.muezzinoglu@gmail.com](mailto:tulay.muezzinoglu@gmail.com) wrote: > Hi, > > I have 3 ubuntu VM's on a private network, each has 64GB ram. I started > ES Beta2 (need it to use term vector feature) on each node with 30GB heap > space and with the following changes in the configuration file: > > discovery.zen.ping.multicast.enabled: false > discovery.zen.ping.unicast.hosts: ["host1","host2","host3"] > bootstrap.mlockall: true > gateway.type: local > gateway.recover\_after\_nodes: 2 > gateway.expected\_nodes: 3 > discovery.zen.minimum\_master\_nodes: 2 > > Sometimes es processes get killed by OOM killer and when I restart a node > I ran into situation like > > > host1,host2,host3 thinking all three nodes are connected and form a > > cluster, and the host3 thinks it is all by itself with status red (names > > are random) > > host1 and host2 form one cluster and host2 and host3 forms another > > cluster. > > Any idea about what I am doing wrong and any suggestions? I will need to > index 20 million documents and have 3 separate indexes with 5 shards 1 > replica. > > Thanks, > T > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/62130b83-40aa-468e-9449-0046497d39ff%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/62130b83-40aa-468e-9449-0046497d39ff%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM\_1C-pbsffn4pCkG5dNXKgJrvKEKF-SsBxoCR3cfRuA7A%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_1C-pbsffn4pCkG5dNXKgJrvKEKF-SsBxoCR3cfRuA7A%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Tula](https://avatars.discourse-cdn.com/v4/letter/t/838e76/32.png) [@Tula](https://discuss.elastic.co/u/Tula)\ **Post date:** [February 22, 2014, 2:31am UTC](https://discuss.elastic.co/t/cluster-setup/15275/3 "2014-02-22T02:31:29Z")
Alex, It looks like the issue was indeed not with the es itself but with our virtual machine setup. We had to setup so that at least 2x heap space reserved to our machines all the time. It has been running okay for a month. Thanks, T On Thursday, January 16, 2014 8:50:38 AM UTC-8, Tula wrote: > Hi, > > I have 3 ubuntu VM's on a private network, each has 64GB ram. I started > ES Beta2 (need it to use term vector feature) on each node with 30GB heap > space and with the following changes in the configuration file: > > discovery.zen.ping.multicast.enabled: false > discovery.zen.ping.unicast.hosts: ["host1","host2","host3"] > bootstrap.mlockall: true > gateway.type: local > gateway.recover\_after\_nodes: 2 > gateway.expected\_nodes: 3 > discovery.zen.minimum\_master\_nodes: 2 > > Sometimes es processes get killed by OOM killer and when I restart a node > I ran into situation like > > > host1,host2,host3 thinking all three nodes are connected and form a > > cluster, and the host3 thinks it is all by itself with status red (names > > are random) > > host1 and host2 form one cluster and host2 and host3 forms another > > cluster. > > Any idea about what I am doing wrong and any suggestions? I will need to > index 20 million documents and have 3 separate indexes with 5 shards 1 > replica. > > Thanks, > T -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1265e27a-6a02-40b8-b2d4-286b96c10429%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1265e27a-6a02-40b8-b2d4-286b96c10429%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:48am UTC](https://discuss.elastic.co/t/cluster-setup/15275/4 "2017-07-06T01:48:10Z")
Closed on July 6, 2017, 1:48am UTC