I get the elusive "Failed!" message when
service elasticsearch start is run on any instance in the cluster if the yml file includes 3 "long" hostnames in the zen.hosts string.
Just to make sure I'm using zen.hosts correctly, using the example
elasticsearch1-elasticsearch-#-vm, I would put the following in the yml file:
discovery.zen.ping.unicast.hosts: ["elasticsearch1-elasticsearch-1-vm", "elasticsearch1-elasticsearch-2-vm", "elasticsearch1-elasticsearch-3-vm"]
I am NOT listing port numbers. I didn't before and never had an issue.
I have no problems starting elasticsearch when the hostname is short and /_cluster/health shows 3 data nodes with a green status.
I do not get any error logs entries in the log files in /var/log/elasticsearch/*
To give you some more background, I'm updating the click to deploy feature on the Google Console and we now support multiple deployments per project. As a result we ask the user for a deployment name and autopopulate the name elasticsearch
n. We insert the deployment name as a prefix to the instance name to avoid instance name conflicts. Therefore the deployment name elasticsearch1 will create the following instances:
This has never been an issue with the current click to deploy version because we capped instance names to about ~20 characters and there was no prefix. this is what's leading me to think the hostname needs to be shorter.
Let me know if I can help clarify further.