I have two clusters, i would like to connect them using remote cluster. I want to connect 2nd to 1st using front address, but internal communication of 1st cluster should be on backend address. I tried with transport.publish_host and transport.bind_host but it doesnt work. When i set
transport.host: [backend_address, front_address], frontend is set everywhere so I have remote cluster connection but communication of this node with another from the same cluster is on frontend too. I don't want that.
Do you have any idea, how to set two different hosts? Sth with profiles?
Just to triple check... do you have two nodes (one local, one remote) or two clusters? I'm assuming clusters given the rest of your post but best check.
When i set
This is just for communication inside the cluster, you can't connect 2 clusters with this setting. Effectively you're trying to somehow make one cluster made up of all the local and remote nodes, which is not going to work well.
Try going through https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-remote-clusters.html again. Remember
transport.host is "local" to each cluster, and the settings on that docs page are for making the two clusters talk to each other.
I have two clusters, from the second one i create remote cluster and i set as a seed one ip address of node from first cluster.
To be honest, I have no idea what's the correct config, in docs they said that remote cluster use transport layer, so the same as internal communication. another tip?
Right, so you want two clusters set up for cross-cluster search. Can you post your whole config for both clusters? You can anonymise IP addresses and so on.
Here is the config:
cluster.name: ABC-XYZ node.name: srv24246jm.ABC.XYZ node.master: false node.data: true node.ingest: true node.attr.type: warm node.attr.server_name: srv24246jm cluster.routing.allocation.awareness.attributes: server_name path.data: ["/var/data/elasticsearch/1","/var/data/elasticsearch/2","/var/data/elasticsearch/3","/var/data/elasticsearch/4","/var/data/elasticsearch/5"] path.logs: /var/log/elasticsearch bootstrap.memory_lock: true network.host: [srv24246jm.ABC.XYZ, 10.234.189.1, 10.250.156.19] #transport.host: [srv24246jm.ABC.XYZ, 10.234.189.1] transport.host: srv24246jm.ABC.XYZ transport.tcp.port: 9300 #http.host: srv24246jm.ABC.XYZ #http.host: 10.234.189.1 http.port: 9200 discovery.zen.minimum_master_nodes: 2 #thread_pool.write.queue_size: 300 #thread_pool.index.queue_size: 300 #thread_pool.search.min_queue_size: 2000 #thread_pool.search.max_queue_size: 2000 discovery.zen.ping.unicast.hosts: ["srv24246jm.ABC.XYZ", "srv24246jn.ABC.XYZ", "srv24246jo.ABC.XYZ", "elk-masternode-1.ABC.XYZ", "elk-masternode-2.ABC.XYZ", "elk-masternode-3.ABC.XYZ", "elk-datanode-1.ABC.XYZ", "elk-datanode-2.ABC.XYZ", "elk-datanode-3.ABC.XYZ", "elk-datanode-4.ABC.XYZ","elk-datanode-5.ABC.XYZ","elk-datanode-6.ABC.XYZ","elk-datanode-7.ABC.XYZ","elk-datanode-8.ABC.XYZ","elk-datanode-9.ABC.XYZ","elk-datanode-10.ABC.XYZ","elk-datanode-11.ABC.XYZ","elk-datanode-12.ABC.XYZ","elk-datanode-13.ABC.XYZ","elk-datanode-14.ABC.XYZ","elk-datanode-15.ABC.XYZ","elk-datanode-16.ABC.XYZ","elk-datanode-17.ABC.XYZ","elk-datanode-18.ABC.XYZ","elk-datanode-19.ABC.XYZ","elk-datanode-20.ABC.XYZ","elk-datanode-21.ABC.XYZ","elk-datanode-22.ABC.XYZ","elk-datanode-23.ABC.XYZ","elk-datanode-24.ABC.XYZ"] xpack.security.enabled: false xpack.monitoring.collection.enabled: true searchguard.enterprise_modules_enabled: true searchguard.allow_unsafe_democertificates: false searchguard.ssl.transport.pemcert_filepath: certs/srv24246jm.ABC.XYZ.pem searchguard.ssl.transport.pemkey_filepath: certs/srv24246jm.ABC.XYZ.key searchguard.ssl.transport.pemtrustedcas_filepath: certs/root-ca.pem searchguard.ssl.transport.enforce_hostname_verification: false searchguard.ssl.http.enabled: true searchguard.ssl.http.pemcert_filepath: certs/srv24246jm.ABC.XYZ.pem searchguard.ssl.http.pemkey_filepath: certs/srv24246jm.ABC.XYZ.key searchguard.ssl.http.pemtrustedcas_filepath: certs/root-ca.pem searchguard.allow_default_init_sgindex: true searchguard.authcz.admin_dn: - CN=t_admin searchguard.nodes_dn: - CN=eskib1.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z - CN=srv24246jm.ABC.XYZ - CN=srv24246jn.ABC.XYZ - CN=srv24246jo.ABC.XYZ - CN=elk-cordnode-1.ABC.XYZ - CN=elk-cordnode-2.ABC.XYZ - CN=elk-cordnode-3.ABC.XYZ - CN=elk-masternode-1.ABC.XYZ - CN=elk-masternode-2.ABC.XYZ - CN=elk-masternode-3.ABC.XYZ - CN=elk-datanode-1.ABC.XYZ - CN=elk-datanode-2.ABC.XYZ - CN=elk-datanode-3.ABC.XYZ - CN=elk-datanode-4.ABC.XYZ - CN=elk-datanode-5.ABC.XYZ - CN=elk-datanode-6.ABC.XYZ - CN=elk-datanode-7.ABC.XYZ - CN=elk-datanode-8.ABC.XYZ - CN=elk-datanode-9.ABC.XYZ - CN=elk-datanode-10.ABC.XYZ - CN=elk-datanode-11.ABC.XYZ - CN=elk-datanode-12.ABC.XYZ - CN=elk-datanode-13.ABC.XYZ - CN=elk-datanode-14.ABC.XYZ - CN=elk-datanode-15.ABC.XYZ - CN=elk-datanode-16.ABC.XYZ - CN=elk-datanode-17.ABC.XYZ - CN=elk-datanode-18.ABC.XYZ - CN=elk-datanode-19.ABC.XYZ - CN=elk-datanode-20.ABC.XYZ - CN=elk-datanode-21.ABC.XYZ - CN=elk-datanode-22.ABC.XYZ - CN=elk-datanode-23.ABC.XYZ - CN=elk-datanode-24.ABC.XYZ searchguard.audit.type: internal_elasticsearch searchguard.enable_snapshot_restore_privilege: true searchguard.check_snapshot_restore_write_privileges: true searchguard.restapi.roles_enabled: ["sg_all_access"] #searchguard.ssl.http.enabled_ciphers: #- "TLS_DHE_RSA_WITH_AES_256_CBC_SHA" #- "TLS_DHE_DSS_WITH_AES_128_CBC_SHA256" searchguard.ssl.http.enabled_protocols: - "TLSv1.1" - "TLSv1.2" searchguard.ssl.transport.enabled_protocols: - "TLSv1.1" - "TLSv1.2" searchguard.audit.config.disabled_rest_categories: NONE searchguard.audit.config.disabled_transport_categories: NONE searchguard.audit.ignore_users: - "logstash" - "remote_monitoring" searchguard.audit.ignore_requests: ["indices:admin/create", "indices:admin/mapping/put"] searchguard.audit.threadpool.size: 0
from this cluster i want to connect to jm:
cluster.name: "oss-elk" network.host: 0.0.0.0 node.name: kibana1 node.master: true node.data: true node.ingest: true path.data: ["/var/data/elasticsearch"] path.logs: /var/log/elasticsearch bootstrap.memory_lock: true #discovery.type: single-node cluster.initial_master_nodes: ["192.168.1.4", "192.168.1.5", "192.168.1.6"] discovery.seed_hosts: ["192.168.1.3","192.168.1.4", "192.168.1.5", "192.168.1.6"] xpack.security.enabled: false xpack.security.transport.ssl.enabled: false xpack.monitoring.collection.enabled: true xpack.monitoring.elasticsearch.collection.enabled: false ######## Start Search Guard Configuration ######## searchguard.ssl.transport.pemcert_filepath: certs/eskib1.pem searchguard.ssl.transport.pemkey_filepath: certs/eskib1.key searchguard.ssl.transport.pemtrustedcas_filepath: certs/osselk-ca.pem searchguard.ssl.transport.enforce_hostname_verification: false searchguard.ssl.transport.resolve_hostname: false searchguard.ssl.http.enabled: true searchguard.ssl.http.pemcert_filepath: certs/eskib1.pem searchguard.ssl.http.pemkey_filepath: certs/eskib1.key searchguard.ssl.http.pemtrustedcas_filepath: certs/osselk-ca.pem searchguard.nodes_dn: - CN=srv24246jm.ABC.XYZ - CN=eskib1.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z - CN=esnode1.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z - CN=esnode2.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z - CN=esnode3.XYZ,OU=CA,O=OSS Team\, Inc.,DC=XY,DC=Z searchguard.authcz.admin_dn: - CN=o_admin searchguard.audit.type: internal_elasticsearch searchguard.enable_snapshot_restore_privilege: true searchguard.check_snapshot_restore_write_privileges: true searchguard.restapi.roles_enabled: ["SGS_ALL_ACCESS"] #cluster.routing.allocation.disk.threshold_enabled: false #node.max_local_storage_nodes: 3
I see that you are using SearchGuard, which might complicate this. I have never used it so am not sure whether it requires any special settings or has any impact on cross-cluster search. I would recommend you either try setting it up without SearchGuard (in which case we should be able to help) or contact SearchGuard to see if they can help out.
Let's skip sg or xpack, because i think sg config is ok, i created same user on both clusters and i mapped roles for them on a remote cluster. I exchanged certs both clusters. During my not correct cross connection permission was ok and i could see and query indices from remote cluster. Let's talk about elastic config, how to set network, communication etc. Is there any require type of node for node from remote cluster? i cant find any example. As i understand cross cluster use transport layer so 9300(default), how to set different host for internal communication and the other one for remote cluster?
What versions are your clusters running? Do they meet the remote cluster version requirements?
Yes, of course. They are compatible, one is 7.6.0(local) and the second 6.8.3 (remote).
This is very important for me and necessary before upgrade to 7.X version. I have to rewrite all alarms from sentinl (which is not supported for 7 version) to Signal and maintain business continuity.
I do not think you can expose different IPs/interfaces the way you describe and separate local and remote transport traffic. Maybe you could add a few dedicated coordinating-only nodes with a different config and connect through these?
probably this is my last option, how about efficiency in this case when node will communicate with others from the same cluster on front address? Won't it be a reason of some problems and affect connection errors?
I'll try it, create one little cord node, or maybe i would do it on existing cord node, because right now i have 3 cord nodes which are connected to 3 kibana instances on the same hosts.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.