I have two nodes up in ec2 configured as follows:
discovery:
type: ec2
initial_state_timeout: 120s
ec2:
host_type: public_dns
ports: 9300
groups: dev_private
I cannot connect to them running tests from nodes on my dev machine, trying to connect as follows:
NodeBuilder nb = NodeBuilder.nodeBuilder().client(true);
nb.settings().put("discovery.zen.ping.multicast.enabled", "false");
nb.settings().put("discovery.zen.ping.unicast.hosts", "elasticsearch1.dev5.cloud.test.com:9300,elasticsearch2.dev5.cloud.test.com:9300");
nb.settings().put("cluster.name", "dev5");
this.indexClient = nb.node().client();
It still seems to try to join the nodes on private ip (the ip mentioned below, 10.198.109.171, is a private ip for one of the nodes):
12:42:14,757 WARN [elasticsearch[Fury][cached]-pool-3-thread-1] [discovery.zen] - [Fury] failed to connect to master [[Freakshow][594cc07f-6181-4794-87c5-7471ba31e70d][inet[/10.198.109.171:9300]]], retrying...
org.elasticsearch.transport.ConnectTransportException: [Freakshow][inet[/10.198.109.171:9300]]
Caused by: org.elasticsearch.transport.ConnectTransportException: [Freakshow][inet[/10.198.109.171:9300]]
12:42:19,275 WARN [elasticsearch[Fury][cached]-pool-3-thread-1] [discovery.zen] - [Fury] failed to connect to master [[Freakshow][594cc07f-6181-4794-87c5-7471ba31e70d][inet[/10.198.109.171:9300]]], retrying...
It does that over and over until it gives up:
12:42:39,845 WARN [main] [discovery] - [Fury] waited for 30s and no initial state was set by the discovery
Can you check in the logs of the nodes what is the publish_host? This is
the one you want if you want to be able to connect to them. Currently, the
ec2 discovery is not tied to the definition of what is the host the nodes
will be connected to, so you will need to configure it as well.
I have two nodes up in ec2 configured as follows:
discovery:
type: ec2
initial_state_timeout: 120s
ec2:
host_type: public_dns
ports: 9300
groups: dev_private
I cannot connect to them running tests from nodes on my dev machine, trying
to connect as follows:
NodeBuilder nb = NodeBuilder.nodeBuilder().client(true);
nb.settings().put("discovery.zen.ping.multicast.enabled",
"false");
nb.settings().put("discovery.zen.ping.unicast.hosts",
"elasticsearch1.dev5.cloud.test.com:9300, elasticsearch2.dev5.cloud.test.com:9300");
nb.settings().put("cluster.name", "dev5");
this.indexClient = nb.node().client();
It still seems to try to join the nodes on private ip (the ip mentioned
below, 10.198.109.171, is a private ip for one of the nodes):
12:42:14,757 WARN [elasticsearch[Fury][cached]-pool-3-thread-1]
[discovery.zen] - [Fury] failed to connect to master
[[Freakshow][594cc07f-6181-4794-87c5-7471ba31e70d][inet[/10.198.109.171:9300
]]],
retrying...
org.elasticsearch.transport.ConnectTransportException:
[Freakshow][inet[/10.198.109.171:9300]]
Caused by: org.elasticsearch.transport.ConnectTransportException:
[Freakshow][inet[/10.198.109.171:9300]]
12:42:19,275 WARN [elasticsearch[Fury][cached]-pool-3-thread-1]
[discovery.zen] - [Fury] failed to connect to master
I'm sorry, I'm not sure I understand your response, but at any rate I think I failed for two reasons to give you a correct understanding of my situation.
We are trying to connect to two ec2-hosted elasticsearch nodes from outside the cloud. We don't have access to ec2's private IP space from our dev network, so we need the elasticsearch discovery to publish the public dns name of the instances (eg ec2-204-236-195-66.compute-1.amazonaws.com), not an IP address. That public DNS name is the only way to access the elasticsearch nodes from both
inside and outside the cloud. My dev machine outside the cloud is the one that produced the ConnectTransportException log errors copied above.
Also, the config of the nodes in the ec2 cloud I gave you was wrong. This is what I should have given you:
discovery:
host_type: public_dns
type: zen
zen:
ping:
unicast:
hosts: elasticsearch1.dev5.cloud.test.com[9300],elasticsearch2.dev5.cloud.test.com[9300]
enabled: true
multicast:
enabled: false
I'm sorry, I'm not sure I understand your response, but at any rate I think
I
failed for two reasons to give you a correct understanding of my situation.
We are trying to connect to two ec2-hosted elasticsearch nodes from outside
the cloud. We don't have access to ec2's private IP space from our dev
network, so we need the elasticsearch discovery to publish the public dns
name of the instances (eg ec2-204-236-195-66.compute-1.amazonaws.com), not
an IP address. That public DNS name is the only way to access the
elasticsearch nodes from both
inside and outside the cloud. My dev machine outside the cloud is the one
that produced the ConnectTransportException log errors copied above.
Also, the config of the nodes in the ec2 cloud I gave you was wrong. This
is what I should have given you:
discovery:
host_type: public_dns
type: zen
zen:
ping:
unicast:
hosts: elasticsearch1.dev5.cloud.test.com[9300], elasticsearch2.dev5.cloud.test.com[9300]
enabled: true
multicast:
enabled: false
Still can't connect, but the errors manifest differently.
I switched to connect the node in my dev machine (outside the ec2 cloud) this way:
indexClient = new TransportClient()
.addTransportAddress(new InetSocketTransportAddress("", 9300))
.addTransportAddress(new InetSocketTransportAddress("", 9300));
I can get through the above lines without exception, but I see in the logs:
16:24:47,658 WARN [main] [client.transport] - [Rodstvow] Node [Jimmy Woo][06aff47a-c269-4ac0-9170-a64d4a66ced1][inet[/10.206.98.255:9300]] not part of the cluster Cluster [elasticsearch], ignoring...
16:24:47,660 WARN [main] [client.transport] - [Rodstvow] Node [Freakshow][594cc07f-6181-4794-87c5-7471ba31e70d][inet[/10.198.109.171:9300]] not part of the cluster Cluster [elasticsearch], ignoring...
10.206.98.255 and 10.198.109.171 are still the private ip's only visible inside the ec2 cloud. Looking the logs of the nodes in the cloud, I don't see any evidence that I've joined.
Then, when I search I get a
org.elasticsearch.client.transport.NoNodeAvailableException: No node available
By the way, once I get it to work with the Transport client, how do I set the cluster name?
You need to set the cluster name on the transport client so it can join the
cluster you started. When you construct a new TransportClient instance, you
can pass it settings, in the settings, you can place "cluster.name" with the
cluster name that you started the nodes with.
You won't see the client joining the nodes, as the transport client is not
part of the cluster.
Still can't connect, but the errors manifest differently.
I switched to connect the node in my dev machine (outside the ec2 cloud)
this way:
indexClient = new TransportClient()
.addTransportAddress(new
InetSocketTransportAddress("", 9300))
.addTransportAddress(new
InetSocketTransportAddress("", 9300));
I can get through the above lines without exception, but I see in the logs:
16:24:47,658 WARN [main] [client.transport] - [Rodstvow] Node [Jimmy
Woo][06aff47a-c269-4ac0-9170-a64d4a66ced1][inet[/10.206.98.255:9300]] not
part of the cluster Cluster [elasticsearch], ignoring...
16:24:47,660 WARN [main] [client.transport] - [Rodstvow] Node
[Freakshow][594cc07f-6181-4794-87c5-7471ba31e70d][inet[/
10.198.109.171:9300]]
not part of the cluster Cluster [elasticsearch], ignoring...
10.206.98.255 and 10.198.109.171 are still the private ip's only visible
inside the ec2 cloud. Looking the logs of the nodes in the cloud, I don't
see any evidence that I've joined.
Then, when I search I get a
org.elasticsearch.client.transport.NoNodeAvailableException: No node
available
By the way, once I get it to work with the Transport client, how do I set
the cluster name?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.