Multiple publish_host for nodes communication behind a firewall


we currently have a cluster in version 7.6.0, composed of 28 nodes spread over 2 datacenters :

  • 14 in datacenter A (named es1-A, es2-A, ..., es14-A)
  • 14 in datacenter B (named es1-B, es2-B, ..., es14-B)
    3 nodes are master-elligible:
  • es2-A
  • es3-A
  • es2-B
    The problem is therefore that if datacenter A falls, the cluster will not be able to reform itself just with the nodes of datacenter B, because they will only see the master node es2-B when they need at least 2.

So, we added a master-elligible node on a 3rd datacenter "C" called es1-C.
And we remove the es3-A node from the master-elligible.

The 3 master-elligible nodes therefore become:

  • es2-A
  • es2-B
  • es1-C
    Which will solve our problem.

But I am unable to integrate es1-C to the cluster for the following reasons:

  • Datacenters A and B are on the same network, the nodes communicate with each other with their private hostname / IP.
  • Datacenter C is not on the same network as A and B.
  • es1-C therefore goes through a public firewall to call the nodes of es2-A and es2-B.
  • But the nodes of A and B expect to be called by their private IP (conf "publish_address")
  • So es1-C tries to call them with their private IPs too, but it can't because it doesn't know them ....

See attached diagram.

Here are my questions:

  • Can a node be reachable for other nodes by multiple addresses ?
  • Does a node need to be able to communicate with all the other nodes in the cluster or does it just need to see the master-elligibles ?

And I don't want all the other nodes of datecenter A and B to go through the firewall to connect to each other

Thank you.

No, Elasticsearch needs the network to "look the same" from every node.

Yes, all nodes need connectivity to all other nodes.

Ok thank you for your response.

Elasticsearch doesn't like forming clusters across networks either. Network latency can cause nodes to drop out for eg.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.