Different hosts for local and remote cluster

wedkarz014 · April 6, 2020, 2:54pm

Hi everyone!
I have two clusters, i would like to connect them using remote cluster. I want to connect 2nd to 1st using front address, but internal communication of 1st cluster should be on backend address. I tried with transport.publish_host and transport.bind_host but it doesnt work. When i set transport.host: [backend_address, front_address], frontend is set everywhere so I have remote cluster connection but communication of this node with another from the same cluster is on frontend too. I don't want that.
Do you have any idea, how to set two different hosts? Sth with profiles?

Emanuil · April 7, 2020, 3:02am

Just to triple check... do you have two nodes (one local, one remote) or two clusters? I'm assuming clusters given the rest of your post but best check.

When i set transport.host

This is just for communication inside the cluster, you can't connect 2 clusters with this setting. Effectively you're trying to somehow make one cluster made up of all the local and remote nodes, which is not going to work well.

Try going through Remote clusters | Elasticsearch Guide [8.11] | Elastic again. Remember transport.host is "local" to each cluster, and the settings on that docs page are for making the two clusters talk to each other.

wedkarz014 · April 7, 2020, 4:07pm

I have two clusters, from the second one i create remote cluster and i set as a seed one ip address of node from first cluster.
To be honest, I have no idea what's the correct config, in docs they said that remote cluster use transport layer, so the same as internal communication. another tip?

Emanuil · April 8, 2020, 10:56am

Right, so you want two clusters set up for cross-cluster search. Can you post your whole config for both clusters? You can anonymise IP addresses and so on.

wedkarz014 · April 8, 2020, 11:42am

Here is the config:

First

cluster.name: ABC-XYZ
node.name: srv24246jm.ABC.XYZ
node.master: false
node.data: true
node.ingest: true
node.attr.type: warm
node.attr.server_name: srv24246jm
cluster.routing.allocation.awareness.attributes: server_name
path.data: ["/var/data/elasticsearch/1","/var/data/elasticsearch/2","/var/data/elasticsearch/3","/var/data/elasticsearch/4","/var/data/elasticsearch/5"]
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
network.host: [srv24246jm.ABC.XYZ, 10.234.189.1, 10.250.156.19]
#transport.host: [srv24246jm.ABC.XYZ, 10.234.189.1]
transport.host: srv24246jm.ABC.XYZ
transport.tcp.port: 9300
#http.host: srv24246jm.ABC.XYZ
#http.host: 10.234.189.1
http.port: 9200
discovery.zen.minimum_master_nodes: 2
#thread_pool.write.queue_size: 300
#thread_pool.index.queue_size: 300
#thread_pool.search.min_queue_size: 2000
#thread_pool.search.max_queue_size: 2000
discovery.zen.ping.unicast.hosts: ["srv24246jm.ABC.XYZ", "srv24246jn.ABC.XYZ", "srv24246jo.ABC.XYZ", "elk-masternode-1.ABC.XYZ", "elk-masternode-2.ABC.XYZ", "elk-masternode-3.ABC.XYZ", "elk-datanode-1.ABC.XYZ", "elk-datanode-2.ABC.XYZ", "elk-datanode-3.ABC.XYZ", "elk-datanode-4.ABC.XYZ","elk-datanode-5.ABC.XYZ","elk-datanode-6.ABC.XYZ","elk-datanode-7.ABC.XYZ","elk-datanode-8.ABC.XYZ","elk-datanode-9.ABC.XYZ","elk-datanode-10.ABC.XYZ","elk-datanode-11.ABC.XYZ","elk-datanode-12.ABC.XYZ","elk-datanode-13.ABC.XYZ","elk-datanode-14.ABC.XYZ","elk-datanode-15.ABC.XYZ","elk-datanode-16.ABC.XYZ","elk-datanode-17.ABC.XYZ","elk-datanode-18.ABC.XYZ","elk-datanode-19.ABC.XYZ","elk-datanode-20.ABC.XYZ","elk-datanode-21.ABC.XYZ","elk-datanode-22.ABC.XYZ","elk-datanode-23.ABC.XYZ","elk-datanode-24.ABC.XYZ"]
xpack.security.enabled: false
xpack.monitoring.collection.enabled: true
searchguard.enterprise_modules_enabled: true
searchguard.allow_unsafe_democertificates: false
searchguard.ssl.transport.pemcert_filepath: certs/srv24246jm.ABC.XYZ.pem
searchguard.ssl.transport.pemkey_filepath: certs/srv24246jm.ABC.XYZ.key
searchguard.ssl.transport.pemtrustedcas_filepath: certs/root-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: certs/srv24246jm.ABC.XYZ.pem
searchguard.ssl.http.pemkey_filepath:  certs/srv24246jm.ABC.XYZ.key
searchguard.ssl.http.pemtrustedcas_filepath: certs/root-ca.pem
searchguard.allow_default_init_sgindex: true
searchguard.authcz.admin_dn:
- CN=t_admin
searchguard.nodes_dn:
- CN=eskib1.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z
- CN=srv24246jm.ABC.XYZ
- CN=srv24246jn.ABC.XYZ
- CN=srv24246jo.ABC.XYZ
- CN=elk-cordnode-1.ABC.XYZ
- CN=elk-cordnode-2.ABC.XYZ
- CN=elk-cordnode-3.ABC.XYZ
- CN=elk-masternode-1.ABC.XYZ
- CN=elk-masternode-2.ABC.XYZ
- CN=elk-masternode-3.ABC.XYZ
- CN=elk-datanode-1.ABC.XYZ
- CN=elk-datanode-2.ABC.XYZ
- CN=elk-datanode-3.ABC.XYZ
- CN=elk-datanode-4.ABC.XYZ
- CN=elk-datanode-5.ABC.XYZ
- CN=elk-datanode-6.ABC.XYZ
- CN=elk-datanode-7.ABC.XYZ
- CN=elk-datanode-8.ABC.XYZ
- CN=elk-datanode-9.ABC.XYZ
- CN=elk-datanode-10.ABC.XYZ
- CN=elk-datanode-11.ABC.XYZ
- CN=elk-datanode-12.ABC.XYZ
- CN=elk-datanode-13.ABC.XYZ
- CN=elk-datanode-14.ABC.XYZ
- CN=elk-datanode-15.ABC.XYZ
- CN=elk-datanode-16.ABC.XYZ
- CN=elk-datanode-17.ABC.XYZ
- CN=elk-datanode-18.ABC.XYZ
- CN=elk-datanode-19.ABC.XYZ
- CN=elk-datanode-20.ABC.XYZ
- CN=elk-datanode-21.ABC.XYZ
- CN=elk-datanode-22.ABC.XYZ
- CN=elk-datanode-23.ABC.XYZ
- CN=elk-datanode-24.ABC.XYZ
searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["sg_all_access"]
#searchguard.ssl.http.enabled_ciphers:
  #- "TLS_DHE_RSA_WITH_AES_256_CBC_SHA"
  #- "TLS_DHE_DSS_WITH_AES_128_CBC_SHA256"
searchguard.ssl.http.enabled_protocols:
  - "TLSv1.1"
  - "TLSv1.2"
searchguard.ssl.transport.enabled_protocols:
  - "TLSv1.1"
  - "TLSv1.2"
searchguard.audit.config.disabled_rest_categories: NONE
searchguard.audit.config.disabled_transport_categories: NONE
searchguard.audit.ignore_users:
  - "logstash"
  - "remote_monitoring"
searchguard.audit.ignore_requests: ["indices:admin/create", "indices:admin/mapping/put"]
searchguard.audit.threadpool.size: 0

from this cluster i want to connect to jm:

second

cluster.name: "oss-elk"
network.host: 0.0.0.0
node.name: kibana1
node.master: true
node.data: true
node.ingest: true
path.data: ["/var/data/elasticsearch"]
path.logs: /var/log/elasticsearch
bootstrap.memory_lock: true
#discovery.type: single-node
cluster.initial_master_nodes: ["192.168.1.4", "192.168.1.5", "192.168.1.6"]
discovery.seed_hosts: ["192.168.1.3","192.168.1.4", "192.168.1.5", "192.168.1.6"]

xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false

xpack.monitoring.collection.enabled: true
xpack.monitoring.elasticsearch.collection.enabled: false


######## Start Search Guard Configuration ########
searchguard.ssl.transport.pemcert_filepath: certs/eskib1.pem
searchguard.ssl.transport.pemkey_filepath: certs/eskib1.key
searchguard.ssl.transport.pemtrustedcas_filepath: certs/osselk-ca.pem
searchguard.ssl.transport.enforce_hostname_verification: false
searchguard.ssl.transport.resolve_hostname: false
searchguard.ssl.http.enabled: true
searchguard.ssl.http.pemcert_filepath: certs/eskib1.pem
searchguard.ssl.http.pemkey_filepath: certs/eskib1.key
searchguard.ssl.http.pemtrustedcas_filepath: certs/osselk-ca.pem
searchguard.nodes_dn:
- CN=srv24246jm.ABC.XYZ
- CN=eskib1.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z
- CN=esnode1.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z
- CN=esnode2.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z
- CN=esnode3.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z
searchguard.authcz.admin_dn:
- CN=o_admin

searchguard.audit.type: internal_elasticsearch
searchguard.enable_snapshot_restore_privilege: true
searchguard.check_snapshot_restore_write_privileges: true
searchguard.restapi.roles_enabled: ["SGS_ALL_ACCESS"]

#cluster.routing.allocation.disk.threshold_enabled: false
#node.max_local_storage_nodes: 3

wedkarz014 · April 14, 2020, 7:16am

anyone?

Christian_Dahlqvist · April 14, 2020, 7:38am

I see that you are using SearchGuard, which might complicate this. I have never used it so am not sure whether it requires any special settings or has any impact on cross-cluster search. I would recommend you either try setting it up without SearchGuard (in which case we should be able to help) or contact SearchGuard to see if they can help out.

wedkarz014 · April 15, 2020, 12:04am

Let's skip sg or xpack, because i think sg config is ok, i created same user on both clusters and i mapped roles for them on a remote cluster. I exchanged certs both clusters. During my not correct cross connection permission was ok and i could see and query indices from remote cluster. Let's talk about elastic config, how to set network, communication etc. Is there any require type of node for node from remote cluster? i cant find any example. As i understand cross cluster use transport layer so 9300(default), how to set different host for internal communication and the other one for remote cluster?

Christian_Dahlqvist · April 15, 2020, 4:44am

What versions are your clusters running? Do they meet the remote cluster version requirements?

wedkarz014 · April 15, 2020, 6:52am

Yes, of course. They are compatible, one is 7.6.0(local) and the second 6.8.3 (remote).
This is very important for me and necessary before upgrade to 7.X version. I have to rewrite all alarms from sentinl (which is not supported for 7 version) to Signal and maintain business continuity.

Christian_Dahlqvist · April 15, 2020, 8:00am

I do not think you can expose different IPs/interfaces the way you describe and separate local and remote transport traffic. Maybe you could add a few dedicated coordinating-only nodes with a different config and connect through these?

wedkarz014 · April 15, 2020, 8:50am

probably this is my last option, how about efficiency in this case when node will communicate with others from the same cluster on front address? Won't it be a reason of some problems and affect connection errors?
I'll try it, create one little cord node, or maybe i would do it on existing cord node, because right now i have 3 cord nodes which are connected to 3 kibana instances on the same hosts.

system · May 13, 2020, 8:50am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.