Failed to send join request to master

Elastic Stack Elasticsearch
1

Hey guys,

I'm currently working with an ES (0.90.10) cluster of six nodes, hosted on
EC2. It worked fine for most of time, but sometimes nodes are missing from
cluster. I restarted the node missing from cluster, and got following log
messages, which suggest the node couldn't connect to the master. I tried to
telnet the master on port 9300, 9200, and 9500, all worked fine. Any idea
what's going on? Or how can I better troubleshoot this problem?

Thanks in advance!!!

[2014-03-01 17:37:04,377][DEBUG][discovery.ec2 ]
[prod-insights-es-3] using dynamic discovery nodes
[[#cloud-i-4e0e9f60-0][inet[/10.238.151.xxx:9300]],
[#cloud-i-846afaaa-0][inet[/10.4.159.xxx:9300]],
[#cloud-i-f2fa69dc-0][inet[/10.225.21.xxx:9300]],
[#cloud-i-db841ff5-0][inet[/10.138.38.xxx:9300]],
[#cloud-i-76b82358-0][inet[/10.141.169.xxx:9300]],
[#cloud-i-2eb12a00-0][inet[/10.225.31.xxx:9300]]]
[2014-03-01 17:37:04,390][DEBUG][discovery.ec2 ]
[prod-insights-es-3] filtered ping responses: (filter_client[true],
filter_data[false])
--> target
[[prod-insights-es-4][VxPFIFgTRQKjKE6UdV8r5w][inet[/10.141.169.xxx:9300]]],
master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]]
--> target
[[prod-insights-es-0][MtJ8gh1ASwyXduy9IDzZXg][inet[/10.238.151.xxx:9300]]],
master [null]
--> target
[[prod-insights-es-2][lQu3NC_QTuybRYb6JV0saA][inet[/10.225.21.xxx:9300]]],
master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]]
--> target
[[prod-insights-es-1][RkEb00GxTY-4pk481I0hNg][inet[/10.4.159.xxx:9300]]],
master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]]
--> target
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]],
master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]]
[2014-03-01 17:37:19,391][INFO ][discovery.ec2 ]
[prod-insights-es-3] failed to send join request to master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]],
reason [org.elasticsearch.ElasticSearchTimeoutException: Timeout waiting
for task.]

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d36c6de8-0634-4978-a283-1868b8211f33%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Try adding network.public_host: to your elasticsearch.yml
file. It worked for me. I'm using CloudFormation, so finding out what IP
address has is not difficult.

On Saturday, March 1, 2014 11:45:41 AM UTC-6, Tian Chu wrote:

Hey guys,

I'm currently working with an ES (0.90.10) cluster of six nodes, hosted on
EC2. It worked fine for most of time, but sometimes nodes are missing from
cluster. I restarted the node missing from cluster, and got following log
messages, which suggest the node couldn't connect to the master. I tried to
telnet the master on port 9300, 9200, and 9500, all worked fine. Any idea
what's going on? Or how can I better troubleshoot this problem?

Thanks in advance!!!

[2014-03-01 17:37:04,377][DEBUG][discovery.ec2 ]
[prod-insights-es-3] using dynamic discovery nodes
[[#cloud-i-4e0e9f60-0][inet[/10.238.151.xxx:9300]],
[#cloud-i-846afaaa-0][inet[/10.4.159.xxx:9300]],
[#cloud-i-f2fa69dc-0][inet[/10.225.21.xxx:9300]],
[#cloud-i-db841ff5-0][inet[/10.138.38.xxx:9300]],
[#cloud-i-76b82358-0][inet[/10.141.169.xxx:9300]],
[#cloud-i-2eb12a00-0][inet[/10.225.31.xxx:9300]]]
[2014-03-01 17:37:04,390][DEBUG][discovery.ec2 ]
[prod-insights-es-3] filtered ping responses: (filter_client[true],
filter_data[false])
--> target
[[prod-insights-es-4][VxPFIFgTRQKjKE6UdV8r5w][inet[/10.141.169.xxx:9300]]],
master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]]
--> target
[[prod-insights-es-0][MtJ8gh1ASwyXduy9IDzZXg][inet[/10.238.151.xxx:9300]]],
master [null]
--> target
[[prod-insights-es-2][lQu3NC_QTuybRYb6JV0saA][inet[/10.225.21.xxx:9300]]],
master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]]
--> target
[[prod-insights-es-1][RkEb00GxTY-4pk481I0hNg][inet[/10.4.159.xxx:9300]]],
master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]]
--> target
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]],
master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]]
[2014-03-01 17:37:19,391][INFO ][discovery.ec2 ]
[prod-insights-es-3] failed to send join request to master
[[prod-insights-es-5][1L19uMCpRgy6lTLU7l30fg][inet[/10.225.31.xxx:9300]]],
reason [org.elasticsearch.ElasticSearchTimeoutException: Timeout waiting
for task.]

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cf1c5d51-a254-4d23-b316-252fb64750c5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.