I'm trying to use the transport client's sniff while accessing a
cluster. I'm hitting the following two issues:
when running with the cluster in the cloud (AWS ec2), I am unable
to connect with client.transport.sniff set to true. there is no
exception thrown at runtime, it doesn't populate the nodelist
if I remove this setting, it connects as expected.
if I run with the cluster running locally, my client connects and
lists all nodes in the cluster
do I need to specify something specifically for sniff to work in a
cloud env?
running with sniff set to true does not provide a durable
connection if the discovery node is killed
if I connect to a running cluster with sniff set to true, wait for
the client to find all nodes in the cluster then terminate the
original node, the client loses its connection. I expected the
connection to fail over to one of the sniffed addresses.
if I list two nodes explicitly, I can terminate either of the two
listed nodes and remain connected to the cluster
I'm not sure if the Transport.Client is supposed to remain
connected if the single discovery node is termed, but it seems like it
could.
When using sniff, the address used to connect to the nodes being added is the "publish_address" (for transport) they start with (while with a non sniff option, its the address you specify). Maybe there is a problem with the publish address that got decided for the nodes (you can see it logged when a node starts up).
Yea, sniff will always rely on the seed of addresses you specify, and if all nodes are down, then nothing will be "sniffed". It can be enhanced to try and use sniffed nodes in this case, and continue to try and figure out the cluster. Shouldn't be too difficult to implement if you are up to it (check TransportClientNodesService). pull requests are welcomed
-shay.banon
On Friday, June 10, 2011 at 2:54 AM, Pat Christopher wrote:
Hello,
Version: 0.16.2, client and server
I'm trying to use the transport client's sniff while accessing a
cluster. I'm hitting the following two issues:
when running with the cluster in the cloud (AWS ec2), I am unable
to connect with client.transport.sniff set to true. there is no
exception thrown at runtime, it doesn't populate the nodelist
if I remove this setting, it connects as expected.
if I run with the cluster running locally, my client connects and
lists all nodes in the cluster
do I need to specify something specifically for sniff to work in a
cloud env?
running with sniff set to true does not provide a durable
connection if the discovery node is killed
if I connect to a running cluster with sniff set to true, wait for
the client to find all nodes in the cluster then terminate the
original node, the client loses its connection. I expected the
connection to fail over to one of the sniffed addresses.
if I list two nodes explicitly, I can terminate either of the two
listed nodes and remain connected to the cluster
I'm not sure if the Transport.Client is supposed to remain
connected if the single discovery node is termed, but it seems like it
could.
When using sniff, the address used to connect to the nodes being added is the "publish_address" (for transport) they start with (while with a non sniff option, its the address you specify). Maybe there is a problem with the publish address that got decided for the nodes (you can see it logged when a node starts up).
Yea, sniff will always rely on the seed of addresses you specify, and if all nodes are down, then nothing will be "sniffed". It can be enhanced to try and use sniffed nodes in this case, and continue to try and figure out the cluster. Shouldn't be too difficult to implement if you are up to it (check TransportClientNodesService). pull requests are welcomed
-shay.banon
On Friday, June 10, 2011 at 2:54 AM, Pat Christopher wrote:
Hello,
Version: 0.16.2, client and server
I'm trying to use the transport client's sniff while accessing a
cluster. I'm hitting the following two issues:
when running with the cluster in the cloud (AWS ec2), I am unable
to connect with client.transport.sniff set to true. there is no
exception thrown at runtime, it doesn't populate the nodelist
if I remove this setting, it connects as expected.
if I run with the cluster running locally, my client connects and
lists all nodes in the cluster
do I need to specify something specifically for sniff to work in a
cloud env?
running with sniff set to true does not provide a durable
connection if the discovery node is killed
if I connect to a running cluster with sniff set to true, wait for
the client to find all nodes in the cluster then terminate the
original node, the client loses its connection. I expected the
connection to fail over to one of the sniffed addresses.
if I list two nodes explicitly, I can terminate either of the two
listed nodes and remain connected to the cluster
I'm not sure if the Transport.Client is supposed to remain
connected if the single discovery node is termed, but it seems like it
could.
Right, that was the main reason why the non sniff option is there at all, to allow for you on the client code to control the IPs to connect to without "sniffing".
On Friday, June 10, 2011 at 3:35 AM, Pat Christopher wrote:
ES is binding the publish_address to the AWS internal ip, which I
can't connect to from my local machine. I'll try my next test from an
AWS node.
On my list! I'll take a look at the TransportClientNodesService
tomorrow.
When using sniff, the address used to connect to the nodes being added is the "publish_address" (for transport) they start with (while with a non sniff option, its the address you specify). Maybe there is a problem with the publish address that got decided for the nodes (you can see it logged when a node starts up).
Yea, sniff will always rely on the seed of addresses you specify, and if all nodes are down, then nothing will be "sniffed". It can be enhanced to try and use sniffed nodes in this case, and continue to try and figure out the cluster. Shouldn't be too difficult to implement if you are up to it (check TransportClientNodesService). pull requests are welcomed
-shay.banon
On Friday, June 10, 2011 at 2:54 AM, Pat Christopher wrote:
Hello,
Version: 0.16.2, client and server
I'm trying to use the transport client's sniff while accessing a
cluster. I'm hitting the following two issues:
when running with the cluster in the cloud (AWS ec2), I am unable
to connect with client.transport.sniff set to true. there is no
exception thrown at runtime, it doesn't populate the nodelist
if I remove this setting, it connects as expected.
if I run with the cluster running locally, my client connects and
lists all nodes in the cluster
do I need to specify something specifically for sniff to work in a
cloud env?
running with sniff set to true does not provide a durable
connection if the discovery node is killed
if I connect to a running cluster with sniff set to true, wait for
the client to find all nodes in the cluster then terminate the
original node, the client loses its connection. I expected the
connection to fail over to one of the sniffed addresses.
if I list two nodes explicitly, I can terminate either of the two
listed nodes and remain connected to the cluster
I'm not sure if the Transport.Client is supposed to remain
connected if the single discovery node is termed, but it seems like it
could.
Right, that was the main reason why the non sniff option is there at all, to allow for you on the client code to control the IPs to connect to without "sniffing".
On Friday, June 10, 2011 at 3:35 AM, Pat Christopher wrote:
ES is binding the publish_address to the AWS internal ip, which I
can't connect to from my local machine. I'll try my next test from an
AWS node.
On my list! I'll take a look at the TransportClientNodesService
tomorrow.
When using sniff, the address used to connect to the nodes being added is the "publish_address" (for transport) they start with (while with a non sniff option, its the address you specify). Maybe there is a problem with the publish address that got decided for the nodes (you can see it logged when a node starts up).
Yea, sniff will always rely on the seed of addresses you specify, and if all nodes are down, then nothing will be "sniffed". It can be enhanced to try and use sniffed nodes in this case, and continue to try and figure out the cluster. Shouldn't be too difficult to implement if you are up to it (check TransportClientNodesService). pull requests are welcomed
-shay.banon
On Friday, June 10, 2011 at 2:54 AM, Pat Christopher wrote:
Hello,
Version: 0.16.2, client and server
I'm trying to use the transport client's sniff while accessing a
cluster. I'm hitting the following two issues:
when running with the cluster in the cloud (AWS ec2), I am unable
to connect with client.transport.sniff set to true. there is no
exception thrown at runtime, it doesn't populate the nodelist
if I remove this setting, it connects as expected.
if I run with the cluster running locally, my client connects and
lists all nodes in the cluster
do I need to specify something specifically for sniff to work in a
cloud env?
running with sniff set to true does not provide a durable
connection if the discovery node is killed
if I connect to a running cluster with sniff set to true, wait for
the client to find all nodes in the cluster then terminate the
original node, the client loses its connection. I expected the
connection to fail over to one of the sniffed addresses.
if I list two nodes explicitly, I can terminate either of the two
listed nodes and remain connected to the cluster
I'm not sure if the Transport.Client is supposed to remain
connected if the single discovery node is termed, but it seems like it
could.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
