Unicast configuration and Master election


(search) #1

Hello,

I have 20 large machines with 256gb of memory on each and I configured the
nodes in the following way:

Total Nodes: 60 (3 per physical machine)
Memory for each node: 30g
Discovery: Unicast
discovery.zen.ping.unicast.hosts: ["host1[9300-9400], host2[9300-9400],
....................host20[9300-9400]"]
discovery.zen.minimum_master_nodes: 31 (TO PREVENT SPLIT BRAIN)
Total Master+Data Nodes:40, DataOnly Nodes:20

Q1: Is this the right way to setup the Unicast? Or should I just specify
one or two hosts in the "unicast.hosts" list and Start those before
starting the rest of the nodes?

When I configured Unicast hosts with all the available hosts, My cluster
is taking very long time to come up and also some of the shards are getting
stuck at "INITIALIZING" state. Also, some of the nodes don't even come up.
I see some warnings in the logs saying cluster initialization not set after
30seconds.

Q2: In large cluster like this, Is there a benefit in making some of the
nodes just Master with no data? If so, how much memory should I give to
Master only nodes?

Thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/58f4466a-f0e2-4775-9010-a5ac60b019d8%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mark Walkom) #2

Are you just running multiple instances of the elasticsearch binary on each
node?
If yes, what is your network interface on each of the physicals? Chances
are you're saturating the interfaces when bringing all the nodes up at once.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 17 December 2013 11:44, Search User feedworks@gmail.com wrote:

Hello,

I have 20 large machines with 256gb of memory on each and I configured the
nodes in the following way:

Total Nodes: 60 (3 per physical machine)
Memory for each node: 30g
Discovery: Unicast
discovery.zen.ping.unicast.hosts: ["host1[9300-9400], host2[9300-9400],
....................host20[9300-9400]"]
discovery.zen.minimum_master_nodes: 31 (TO PREVENT SPLIT BRAIN)
Total Master+Data Nodes:40, DataOnly Nodes:20

Q1: Is this the right way to setup the Unicast? Or should I just specify
one or two hosts in the "unicast.hosts" list and Start those before
starting the rest of the nodes?

When I configured Unicast hosts with all the available hosts, My cluster
is taking very long time to come up and also some of the shards are getting
stuck at "INITIALIZING" state. Also, some of the nodes don't even come up.
I see some warnings in the logs saying cluster initialization not set after
30seconds.

Q2: In large cluster like this, Is there a benefit in making some of the
nodes just Master with no data? If so, how much memory should I give to
Master only nodes?

Thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/58f4466a-f0e2-4775-9010-a5ac60b019d8%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Y8x1xaTDu93w6qkuLnAXYjpp3Hcz_8%3Df14rtyuOkSxGg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(search) #3

I have Gig eth on all of the nodes and the only software running on each
node is 3 instances of ES from the same binary.

On Tuesday, December 17, 2013 3:32:19 AM UTC-5, Mark Walkom wrote:

Are you just running multiple instances of the elasticsearch binary on
each node?
If yes, what is your network interface on each of the physicals? Chances
are you're saturating the interfaces when bringing all the nodes up at once.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 17 December 2013 11:44, Search User <feed...@gmail.com <javascript:>>wrote:

Hello,

I have 20 large machines with 256gb of memory on each and I configured
the nodes in the following way:

Total Nodes: 60 (3 per physical machine)
Memory for each node: 30g
Discovery: Unicast
discovery.zen.ping.unicast.hosts: ["host1[9300-9400], host2[9300-9400],
....................host20[9300-9400]"]
discovery.zen.minimum_master_nodes: 31 (TO PREVENT SPLIT BRAIN)
Total Master+Data Nodes:40, DataOnly Nodes:20

Q1: Is this the right way to setup the Unicast? Or should I just specify
one or two hosts in the "unicast.hosts" list and Start those before
starting the rest of the nodes?

When I configured Unicast hosts with all the available hosts, My cluster
is taking very long time to come up and also some of the shards are getting
stuck at "INITIALIZING" state. Also, some of the nodes don't even come up.
I see some warnings in the logs saying cluster initialization not set after
30seconds.

Q2: In large cluster like this, Is there a benefit in making some of the
nodes just Master with no data? If so, how much memory should I give to
Master only nodes?

Thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/58f4466a-f0e2-4775-9010-a5ac60b019d8%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3aeba813-89d2-4b4d-9417-134d8957d11e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Mark Walkom) #4

It might be worth starting one instance per physical, then seeing if it
connects. Then start another, and repeat.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 17 December 2013 23:41, Search User feedworks@gmail.com wrote:

I have Gig eth on all of the nodes and the only software running on each
node is 3 instances of ES from the same binary.

On Tuesday, December 17, 2013 3:32:19 AM UTC-5, Mark Walkom wrote:

Are you just running multiple instances of the elasticsearch binary on
each node?
If yes, what is your network interface on each of the physicals? Chances
are you're saturating the interfaces when bringing all the nodes up at once.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com

On 17 December 2013 11:44, Search User feed...@gmail.com wrote:

Hello,

I have 20 large machines with 256gb of memory on each and I configured
the nodes in the following way:

Total Nodes: 60 (3 per physical machine)
Memory for each node: 30g
Discovery: Unicast
discovery.zen.ping.unicast.hosts: ["host1[9300-9400], host2[9300-9400],
....................host20[9300-9400]"]
discovery.zen.minimum_master_nodes: 31 (TO PREVENT SPLIT BRAIN)
Total Master+Data Nodes:40, DataOnly Nodes:20

Q1: Is this the right way to setup the Unicast? Or should I just specify
one or two hosts in the "unicast.hosts" list and Start those before
starting the rest of the nodes?

When I configured Unicast hosts with all the available hosts, My
cluster is taking very long time to come up and also some of the shards are
getting stuck at "INITIALIZING" state. Also, some of the nodes don't even
come up. I see some warnings in the logs saying cluster initialization not
set after 30seconds.

Q2: In large cluster like this, Is there a benefit in making some of the
nodes just Master with no data? If so, how much memory should I give to
Master only nodes?

Thanks.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/58f4466a-f0e2-4775-9010-a5ac60b019d8%
40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3aeba813-89d2-4b4d-9417-134d8957d11e%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bHq5kf1Tj2-2_uamGZLk3t9UqPW23nAez4KOU51V%2BCqQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #5