I try to deploy this configuration:
ES 6.6.2 on several docker hosts, 3 nodes.
Two nodes are master-eligible and one node is "data only".
XPACK enabled (basic license).
Inter node communication is encrypted by "p12" certificate with "xpack.security.transport.ssl.verification_mode=certificate". Actually, I generated one common "p12" certificate for all nodes
docker host1 .env: https://pastebin.com/CEXSMJD0
docker host2 .env: https://pastebin.com/wWvcJ7xn
docker host3 .env: https://pastebin.com/NYKcLSTp
When I run host1's docker-compose and host2's docker-compose - cluster "es2" up an running "green", with "elastic2" as "master" node.
Next step I trying to add additional node "elastic3" as "data only" node.
On master node "elastic2" I can see in logs: https://pastebin.com/mFm1vuDx
On other master eligible node "elastic1" I can see in logs: https://pastebin.com/YG87rmyd
The problem also occured on dockerized ES version 6.7.2. (done it just for test)
If I change "DISCOVERY_ZEN_MINIMUM_MASTER_NODES" to "3" and "NODE_MASTER" to "true" (on host3's .env file)- cluster up and running "green" with 3 nodes. (I don't like this choice because if only one node fall down - cluster cannot elect "master node")
If I will change option "xpack.security.enabled" to "false" in all docker-compose.yml files across all nodes - cluster up and running "green" with 3 nodes. (I don't like this choice because I need some security options from xpack)
If I upgrade version of ES to 6.8.3 (just change version in all docker-compose.yml files) - cluster up and running "green" with 3 nodes. (I cannot use version upper than 6.6.2 because of some breaking changes in perl ES module.)
I am running out of ideas, please help.
With best regards,