Transport and syncing data in ES

Elastic Stack Elasticsearch
,
1

Hello,
I have two elasticsearch clusters. One for acceptance where are 3 nodes (each has data, ingest, master role) and one for production where are 9 nodes (3 master, 3 ingest + hot nodes and 3 warm nodes). I would like to set transport traffic between them through interconnection cable. But I want to achieve that nodes from another cluster can connect by another interface because I want to use Cross Cluster Replication.

ACC:
My config is:

node.name: es-node1
discovery.seed_hosts: X.X.2.101:9300,X.X.2.102:9300
cluster.initial_master_nodes: X.X.2.100,X.X.2.101,X.X.2.102
network.publish_host: Y.Y.19.20
network.bind_host: 0.0.0.0

So I have interconnection with addresses X.X.2.100, X.X.2.101, X.X.2.102, and Addresses that are accessible from Internet Y.Y.19.20, Y.Y.19.21, Y.Y.19.22

Is there a way to achieve that syncing on nodes will be transported through interconnect but there will be way to use CCR from another cluster on Internet Interface?

With settings like I posted, nodes are discovered by interconnect but syncing and communications go through the Internet interface.

And in Production I have an idea
But i couldn’t test it, because I cant afford any downtime.
If I set on masters that transport will be done by all interfaces but on Data nodes I will use only interconnect, will it work? Because masters doesnt need to sync with data nodes? I hope so. And then when I will want to use CCR, I will call request on master that will be accessible.

Thanks for you help. I thought that when I set discovery.seed with interconnect addresses, so they will sync with interconnect, but that was wrong :confused:

2

Which version of Elasticsearch are you using?

3

Transport traffic within a cluster uses the publish addresses of the nodes, so this will send all the internal traffic via the external interface. Instead, you should use the internal addresses for the publish address, and use proxy mode for the remote cluster connections.

You must remove this setting once the cluster has formed for the first time.

3 Likes
4

On production I am running this version.

  "version" : {
    "number" : "7.2.0",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "508c38a",
    "build_date" : "2019-06-20T15:54:18.811730Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

I know I should upgrade, I am working on it but first I need to limit and shape ES recv and trans network usage on switches to be as low as possible and has the biggest network usage on internal connection.

5

Thanks, now my nodes are connecting via internal connection, and only ingest goes threw external.

I commented: cluster.initial_master_nodes: X.X.2.100, X.X.2.101, X.X.2.102, and ES still working, thanks, did not know about that.

I'll try to set up proxy mode and if I don't open another topic, I've succeeded :smiley:
Thanks, I am closing.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.