Hi Everybody,
I have setup an Elasticsearch[8.4] cluster (2 nodes) in my aws ec2 machines as follows:
EC2_1 (private IP: 172.31.1.1/32, docker-bridge range: 172.17.0.1/16)
--> in this ec2 I've installed Kubernetes master node, docker and Elasticsearch Master node
EC2_2 (private IP: 172.31.1.2/32, docker-bridge range: 172.17.0.1/16)
--> in this ec2 I've installed Kubernetes worker node, docker and Elasticsearch Slave node
Kubernetes works fine.
Issue: As soon as I start Elasticsearch (for both nodes) I see the nodes cannot communicate between them. This happens because the nodes, inside the clusters, have the wrong IP addresses.
It seems, Elasticsearch, at startup, takes the docker-bridge range IP instead of the EC2 range IP.
from EC2_2 Elasticsearc logs:
[2022-09-22T08:10:35,691][WARN ][o.e.d.HandshakingTransportAddressConnector] [ip-172-31-1-2] completed handshake with [{ip-172-31-1-1}{ip-172-31-1-1}{172.17.0.1}{172.17.0.1:9300}{cdfhilmrstw}] at [172.31.1.1:9300] but followup connection to [172.17.0.1:9300] failed
org.elasticsearch.transport.ConnectTransportException: [ip-172-31-1-1][172.17.0.1:9300] handshake failed. unexpected remote node {ip-172-31-1-2}{ip-172-31-1-2}{172.17.0.1}{172.17.0.1:9300}
If I try to stop docker.service and start Elasticsearch cluster, everything works fine! (both nodes)
I was wondering if I could set cluster nodes IP at Elasticsearch startup (the same as his EC2).
How can I solve the issue? is It an Elastisearch bug?
Thanks in advance,
Simone