Discovery multicast vs unicast

Elastic Stack Elasticsearch
1

multicast discovery or unicast discovery?

Would someone explain the differences? Primarily in how flexible the
cluster/node design will be. I'm wanting to be able to spin up new
nodes without having to require the java client or other nodes to make
configuration changes (which seems to be what I'd have to do if I ran
the unicast)

Also, multicast has a "group" setting with an ip address. Where does
that ip address come from? Do all nodes use the same group ip address?
Is that group ip address also a node?

Also, multicast has a "address" to bind to setting. I'm unclear as to
how this could be used. The example on the site is setting it to
"null" so that it binds to all available networks. How does binding it
to a network ip address change the behavior of the multicast discovery?

2

I believe multicast is limited to the subnet the machines are on. It works
nice to get a cluster up and working quickly. The drawback is that in order
to have multiple clusters on the same subnet, each cluster would need it's
on name. I'm not sure about the group ip setting.

Unicast allows you to specify which nodes to contact to form a cluster.
Unicast takes a bit more work to manage, but gives you more control as well.

If you have multiple interfaces on a machine or multiple ip address, you
can either specify what address to listen on or have the node listen on all
available addresses.

We were originally using multicast, but switched to unicast locally. We
also use unicast on AWS to make it easier to run multiple clusters.

  • Craig

On Wed, Jan 25, 2012 at 3:31 PM, Wes Plunk wes@wesandemily.com wrote:

multicast discovery or unicast discovery?

Would someone explain the differences? Primarily in how flexible the
cluster/node design will be. I'm wanting to be able to spin up new
nodes without having to require the java client or other nodes to make
configuration changes (which seems to be what I'd have to do if I ran
the unicast)

Also, multicast has a "group" setting with an ip address. Where does
that ip address come from? Do all nodes use the same group ip address?
Is that group ip address also a node?

Also, multicast has a "address" to bind to setting. I'm unclear as to
how this could be used. The example on the site is setting it to
"null" so that it binds to all available networks. How does binding it
to a network ip address change the behavior of the multicast discovery?

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.com http://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

3

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

On Jan 25, 5:54 pm, Craig Brown cbr...@youwho.com wrote:

I believe multicast is limited to the subnet the machines are on. It works
nice to get a cluster up and working quickly. The drawback is that in order
to have multiple clusters on the same subnet, each cluster would need it's
on name. I'm not sure about the group ip setting.

Unicast allows you to specify which nodes to contact to form a cluster.
Unicast takes a bit more work to manage, but gives you more control as well.

If you have multiple interfaces on a machine or multiple ip address, you
can either specify what address to listen on or have the node listen on all
available addresses.

We were originally using multicast, but switched to unicast locally. We
also use unicast on AWS to make it easier to run multiple clusters.

  • Craig

On Wed, Jan 25, 2012 at 3:31 PM, Wes Plunk w...@wesandemily.com wrote:

multicast discovery or unicast discovery?

Would someone explain the differences? Primarily in how flexible the
cluster/node design will be. I'm wanting to be able to spin up new
nodes without having to require the java client or other nodes to make
configuration changes (which seems to be what I'd have to do if I ran
the unicast)

Also, multicast has a "group" setting with an ip address. Where does
that ip address come from? Do all nodes use the same group ip address?
Is that group ip address also a node?

Also, multicast has a "address" to bind to setting. I'm unclear as to
how this could be used. The example on the site is setting it to
"null" so that it binds to all available networks. How does binding it
to a network ip address change the behavior of the multicast discovery?

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.comhttp://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

4

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to the
entire network.

5

Hi,

if you know IP/DNS names of all your current and future servers in the
cluster then you can list all of them into unicast configuration but start
only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you start with
three nodes only. Once you need to kick start a new node, you simply make
sure that it gets one of the remaining IP addresses. In such case you do
not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman nick@deadorange.com wrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to the
entire network.

6

Note also that you don't have to list all the nodes in the unicast list, just enough of them so they can be used to discover each other.

On Thursday, January 26, 2012 at 10:59 AM, Lukáš Vlček wrote:

Hi,

if you know IP/DNS names of all your current and future servers in the cluster then you can list all of them into unicast configuration but start only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you start with three nodes only. Once you need to kick start a new node, you simply make sure that it gets one of the remaining IP addresses. In such case you do not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman <nick@deadorange.com (mailto:nick@deadorange.com)> wrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to the entire network.

7

Shay,

you mean something like this?

Node A unicast to [B]
Node B unicast to [C]
Node C unicast to [A]

And all three nodes when started will form a fully functional cluster?

Regards,
Lukas

On Thu, Jan 26, 2012 at 10:57 AM, Shay Banon kimchy@gmail.com wrote:

Note also that you don't have to list all the nodes in the unicast list,
just enough of them so they can be used to discover each other.

On Thursday, January 26, 2012 at 10:59 AM, Lukáš Vlček wrote:

Hi,

if you know IP/DNS names of all your current and future servers in the
cluster then you can list all of them into unicast configuration but start
only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you start with
three nodes only. Once you need to kick start a new node, you simply make
sure that it gets one of the remaining IP addresses. In such case you do
not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman nick@deadorange.com wrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to the
entire network.

8

And on top of maybe a client that points to Node A and is then able to
discover Nodes B and C?

On Jan 26, 4:07 am, Lukáš Vlček lukas.vl...@gmail.com wrote:

Shay,

you mean something like this?

Node A unicast to [B]
Node B unicast to [C]
Node C unicast to [A]

And all three nodes when started will form a fully functional cluster?

Regards,
Lukas

On Thu, Jan 26, 2012 at 10:57 AM, Shay Banon kim...@gmail.com wrote:

Note also that you don't have to list all the nodes in the unicast list,
just enough of them so they can be used to discover each other.

On Thursday, January 26, 2012 at 10:59 AM, Lukáš Vlček wrote:

Hi,

if you know IP/DNS names of all your current and future servers in the
cluster then you can list all of them into unicast configuration but start
only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you start with
three nodes only. Once you need to kick start a new node, you simply make
sure that it gets one of the remaining IP addresses. In such case you do
not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman n...@deadorange.com wrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to the
entire network.

9

On spin up, the nodes try to find the master I believe, so you would need
the list to include one of the nodes that would be the master. We had some
2 node clusters, then spun up 2 more nodes and configured unicast on the
new nodes to point to the 2 existing nodes. They came up just fine because
one of the existing nodes was guaranteed to be the master. When we rebuilt
the cluster, we included the addresses of all for nodes on each machine
sine any of them could be the master now.

  • Craig

On Thu, Jan 26, 2012 at 3:07 AM, Lukáš Vlček lukas.vlcek@gmail.com wrote:

Shay,

you mean something like this?

Node A unicast to [B]
Node B unicast to [C]
Node C unicast to [A]

And all three nodes when started will form a fully functional cluster?

Regards,
Lukas

On Thu, Jan 26, 2012 at 10:57 AM, Shay Banon kimchy@gmail.com wrote:

Note also that you don't have to list all the nodes in the unicast list,
just enough of them so they can be used to discover each other.

On Thursday, January 26, 2012 at 10:59 AM, Lukáš Vlček wrote:

Hi,

if you know IP/DNS names of all your current and future servers in the
cluster then you can list all of them into unicast configuration but start
only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you start
with three nodes only. Once you need to kick start a new node, you simply
make sure that it gets one of the remaining IP addresses. In such case you
do not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman nick@deadorange.comwrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to
the entire network.

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.com http://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

10

So to achieve sort of a load balancer approach you'll need to have at
least 2 node configurable to be the master. The client api does the
load balancer by looking for a master among the two that are
configured and the first available is used. Self discovery among the
nodes in the cluster work the same way, in that the only config nodes
they need to be aware of are the ones marked as being a master and
then they will discover the other nodes via the master.

Is this right?

On Jan 26, 10:21 am, Craig Brown cbr...@youwho.com wrote:

On spin up, the nodes try to find the master I believe, so you would need
the list to include one of the nodes that would be the master. We had some
2 node clusters, then spun up 2 more nodes and configured unicast on the
new nodes to point to the 2 existing nodes. They came up just fine because
one of the existing nodes was guaranteed to be the master. When we rebuilt
the cluster, we included the addresses of all for nodes on each machine
sine any of them could be the master now.

  • Craig

On Thu, Jan 26, 2012 at 3:07 AM, Lukáš Vlček lukas.vl...@gmail.com wrote:

Shay,

you mean something like this?

Node A unicast to [B]
Node B unicast to [C]
Node C unicast to [A]

And all three nodes when started will form a fully functional cluster?

Regards,
Lukas

On Thu, Jan 26, 2012 at 10:57 AM, Shay Banon kim...@gmail.com wrote:

Note also that you don't have to list all the nodes in the unicast list,
just enough of them so they can be used to discover each other.

On Thursday, January 26, 2012 at 10:59 AM, Lukáš Vlček wrote:

Hi,

if you know IP/DNS names of all your current and future servers in the
cluster then you can list all of them into unicast configuration but start
only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you start
with three nodes only. Once you need to kick start a new node, you simply
make sure that it gets one of the remaining IP addresses. In such case you
do not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman n...@deadorange.comwrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to
the entire network.

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.comhttp://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

11

When working with 2 nodes and starting them at the same time, one of them
will be elected master, then the other will join the cluster. When starting
another node on unicast, say a client node, it needs to know the other
nodes in the cluster and will ask each in turn which one is the master.
Once the master is discovered, then it will join. If you have a cluster and
the master goes down, one of the other nodes will be elected as master.

You can configure only 1 of the 2 nodes to become master if you want to.
The neat thing about the client is that it becomes a node in the cluster
like all of the other nodes, except that it usually is not a data node.
There are some more configuration options that allow you to control master
selection among larger groups of nodes. These settings can also be used to
defeat split-brain problems in the cluster.

  • Craig

On Thu, Jan 26, 2012 at 12:34 PM, Wes Plunk wes@wesandemily.com wrote:

So to achieve sort of a load balancer approach you'll need to have at
least 2 node configurable to be the master. The client api does the
load balancer by looking for a master among the two that are
configured and the first available is used. Self discovery among the
nodes in the cluster work the same way, in that the only config nodes
they need to be aware of are the ones marked as being a master and
then they will discover the other nodes via the master.

Is this right?

On Jan 26, 10:21 am, Craig Brown cbr...@youwho.com wrote:

On spin up, the nodes try to find the master I believe, so you would need
the list to include one of the nodes that would be the master. We had
some
2 node clusters, then spun up 2 more nodes and configured unicast on the
new nodes to point to the 2 existing nodes. They came up just fine
because
one of the existing nodes was guaranteed to be the master. When we
rebuilt
the cluster, we included the addresses of all for nodes on each machine
sine any of them could be the master now.

  • Craig

On Thu, Jan 26, 2012 at 3:07 AM, Lukáš Vlček lukas.vl...@gmail.com
wrote:

Shay,

you mean something like this?

Node A unicast to [B]
Node B unicast to [C]
Node C unicast to [A]

And all three nodes when started will form a fully functional cluster?

Regards,
Lukas

On Thu, Jan 26, 2012 at 10:57 AM, Shay Banon kim...@gmail.com wrote:

Note also that you don't have to list all the nodes in the unicast
list,
just enough of them so they can be used to discover each other.

On Thursday, January 26, 2012 at 10:59 AM, Lukáš Vlček wrote:

Hi,

if you know IP/DNS names of all your current and future servers in the
cluster then you can list all of them into unicast configuration but
start
only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you start
with three nodes only. Once you need to kick start a new node, you
simply
make sure that it gets one of the remaining IP addresses. In such
case you
do not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman <n...@deadorange.com
wrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to
the entire network.

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.comhttp://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.com http://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

12

Thanks Craig, Lukas, Shay & Nick, that's what I was concluding.
Appreciate the insight

On Jan 26, 2:25 pm, Craig Brown cbr...@youwho.com wrote:

When working with 2 nodes and starting them at the same time, one of them
will be elected master, then the other will join the cluster. When starting
another node on unicast, say a client node, it needs to know the other
nodes in the cluster and will ask each in turn which one is the master.
Once the master is discovered, then it will join. If you have a cluster and
the master goes down, one of the other nodes will be elected as master.

You can configure only 1 of the 2 nodes to become master if you want to.
The neat thing about the client is that it becomes a node in the cluster
like all of the other nodes, except that it usually is not a data node.
There are some more configuration options that allow you to control master
selection among larger groups of nodes. These settings can also be used to
defeat split-brain problems in the cluster.

  • Craig

On Thu, Jan 26, 2012 at 12:34 PM, Wes Plunk w...@wesandemily.com wrote:

So to achieve sort of a load balancer approach you'll need to have at
least 2 node configurable to be the master. The client api does the
load balancer by looking for a master among the two that are
configured and the first available is used. Self discovery among the
nodes in the cluster work the same way, in that the only config nodes
they need to be aware of are the ones marked as being a master and
then they will discover the other nodes via the master.

Is this right?

On Jan 26, 10:21 am, Craig Brown cbr...@youwho.com wrote:

On spin up, the nodes try to find the master I believe, so you would need
the list to include one of the nodes that would be the master. We had
some
2 node clusters, then spun up 2 more nodes and configured unicast on the
new nodes to point to the 2 existing nodes. They came up just fine
because
one of the existing nodes was guaranteed to be the master. When we
rebuilt
the cluster, we included the addresses of all for nodes on each machine
sine any of them could be the master now.

  • Craig

On Thu, Jan 26, 2012 at 3:07 AM, Lukáš Vlček lukas.vl...@gmail.com
wrote:

Shay,

you mean something like this?

Node A unicast to [B]
Node B unicast to [C]
Node C unicast to [A]

And all three nodes when started will form a fully functional cluster?

Regards,
Lukas

On Thu, Jan 26, 2012 at 10:57 AM, Shay Banon kim...@gmail.com wrote:

Note also that you don't have to list all the nodes in the unicast
list,
just enough of them so they can be used to discover each other.

On Thursday, January 26, 2012 at 10:59 AM, Lukáš Vlček wrote:

Hi,

if you know IP/DNS names of all your current and future servers in the
cluster then you can list all of them into unicast configuration but
start
only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you start
with three nodes only. Once you need to kick start a new node, you
simply
make sure that it gets one of the remaining IP addresses. In such
case you
do not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman <n...@deadorange.com
wrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node broadcasts to
the entire network.

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.comhttp://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.comhttp://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

13

Sure Wes, good luck!

  • Craig

On Thu, Jan 26, 2012 at 9:09 PM, Wes Plunk wes@wesandemily.com wrote:

Thanks Craig, Lukas, Shay & Nick, that's what I was concluding.
Appreciate the insight

On Jan 26, 2:25 pm, Craig Brown cbr...@youwho.com wrote:

When working with 2 nodes and starting them at the same time, one of them
will be elected master, then the other will join the cluster. When
starting
another node on unicast, say a client node, it needs to know the other
nodes in the cluster and will ask each in turn which one is the master.
Once the master is discovered, then it will join. If you have a cluster
and
the master goes down, one of the other nodes will be elected as master.

You can configure only 1 of the 2 nodes to become master if you want to.
The neat thing about the client is that it becomes a node in the cluster
like all of the other nodes, except that it usually is not a data node.
There are some more configuration options that allow you to control
master
selection among larger groups of nodes. These settings can also be used
to
defeat split-brain problems in the cluster.

  • Craig

On Thu, Jan 26, 2012 at 12:34 PM, Wes Plunk w...@wesandemily.com
wrote:

So to achieve sort of a load balancer approach you'll need to have at
least 2 node configurable to be the master. The client api does the
load balancer by looking for a master among the two that are
configured and the first available is used. Self discovery among the
nodes in the cluster work the same way, in that the only config nodes
they need to be aware of are the ones marked as being a master and
then they will discover the other nodes via the master.

Is this right?

On Jan 26, 10:21 am, Craig Brown cbr...@youwho.com wrote:

On spin up, the nodes try to find the master I believe, so you would
need
the list to include one of the nodes that would be the master. We had
some
2 node clusters, then spun up 2 more nodes and configured unicast on
the
new nodes to point to the 2 existing nodes. They came up just fine
because
one of the existing nodes was guaranteed to be the master. When we
rebuilt
the cluster, we included the addresses of all for nodes on each
machine
sine any of them could be the master now.

  • Craig

On Thu, Jan 26, 2012 at 3:07 AM, Lukáš Vlček lukas.vl...@gmail.com
wrote:

Shay,

you mean something like this?

Node A unicast to [B]
Node B unicast to [C]
Node C unicast to [A]

And all three nodes when started will form a fully functional
cluster?

Regards,
Lukas

On Thu, Jan 26, 2012 at 10:57 AM, Shay Banon kim...@gmail.com
wrote:

Note also that you don't have to list all the nodes in the unicast
list,
just enough of them so they can be used to discover each other.

On Thursday, January 26, 2012 at 10:59 AM, Lukáš Vlček wrote:

Hi,

if you know IP/DNS names of all your current and future servers
in the
cluster then you can list all of them into unicast configuration
but
start
only some of them.

Let's say you reserve 20 IP addresses for whole cluster but you
start
with three nodes only. Once you need to kick start a new node, you
simply
make sure that it gets one of the remaining IP addresses. In such
case you
do not have to reconfigure whole cluster with even new node.

Regards,
Lukas

On Thu, Jan 26, 2012 at 5:15 AM, Nick Hoffman <
n...@deadorange.com
wrote:

On Wednesday, 25 January 2012 22:03:43 UTC-5, Wes Plunk wrote:

So in your environment if you spin up a new instance you do
actually
have to make config changes in code and all nodes?

With unicast, yes. With multicast, no, because each node
broadcasts to
the entire network.

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.comhttp://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.comhttp://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939

--
…
CRAIG BROWN
chief architect
youwho, Inc.

www.youwho.com http://www.youwho.com/

T: 801.855. 0921
M: 801.913. 0939