I have a simple two node cluster, and I'm trying to run a custom Java client (indexer-client-node0) on the same server as one of the 'data' (XX4) nodes. It happens that data-node2 (XX5) is master at the moment - which may to have some relevancy here?
I was able to connect the client previously when there was only one node, and the client and data nodes were on the same server. The problems began when the second data node was added.
Relevant log entries from client:
14:02:36,967 DEBUG [netty:895] [indexer-client-node0] connected to node [{#zen_unicast_1#}{10.10.XXX.XX4}{10.10.XXX.XX4:9300}]
14:02:39,969 DEBUG [netty:1032] [indexer-client-node0] disconnecting from [{#zen_unicast_1#}{10.10.XXX.XX4}{10.10.XXX.XX4:9300}] due to explicit disconnect call
14:02:39,972 INFO [zen:443] [indexer-client-node0] failed to send join request to master [{elastic02}{e29o0UG4SmOWrtEcfs1u2Q}{10.10.XXX.XX5}{10.10.XXX.XX5:9300}], reason [RemoteTransportException[[elastic02][10.10.XXX.XX5:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[indexer-client-node0][127.0.0.1:9301] connect_timeout[30s]]; nested: NotSerializableExceptionWrapper[Connection refused: /127.0.0.1:9301]; ]]
Version: Elasticsearch 2.1.1
Both server elasticsearch.yml have: network.host: _eth0_
One settings I'm not sure about is - discovery.zen.ping.unicast.hosts. I have only one entry on each servers' elasticsearch.yml - and they are pointing to each other. Not quite sure if this is correct or not?
This looks relevant. It seems the client is using port 9301, which is good, but why is it connecting to localhost?
What's your connection string in the client look like?
Yes, it does resolve. Actually the data nodes also refer to one another by domain name. I also (just) switched the client-node to IP referral - appears to be the same results.
To quote from the advise in the page above: "If you’re using Java, the Transport client should be chosen over the Node client unless the performance gain from using a Node client turns out to be large enough to warrant the additional network complexity. Use benchmarks to verify the performance gains."
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.