Configuration of kibana with cross cluster search

[bbailey (Bill Bailey)]

I am switching my multi-cluster installation from using kibana running on a tribe node to using kibana with cross cluster search running on a single standalone elasticsearch node. From the documentation it appears to be recommended to use gateway nodes but there is no clear definition of what a gateway node is.

Old setup
DC1 cluster - 3 data nodes, 3 master nodes, 1 tribe/kibana node
DC2 cluster - 3 data nodes, 3 master nodes
DC3 cluster - 2 data nodes, 3 master nodes

New setup
DC1 cluster - 3 data nodes, 3 master nodes
DC2 cluster - 3 data nodes, 3 master nodes
DC3 cluster - 2 data nodes, 3 master nodes
Kibana - 1 data/master/kibana node

Question 1. Is a gateway node simply a client node ( i.e. node.master:false node.data:false node:ingest:false )

I have currently set up my cross cluster search seeds to be the 3 master nodes in each cluster

Question 2. Should I instead add at least one gateway node to each cluster and configure these as my seeds ?

[warkolm (Mark Walkom)]

@bbailey

Question 1. Is a gateway node simply a client node ( i.e. node.master:false node.data:false node:ingest:false )

It's the node(s) that the cross cluster configuration uses to talk to. It explains it here - Cross-cluster search | Elasticsearch Reference [6.1] | Elastic

@bbailey

Question 2. Should I instead add at least one gateway node to each cluster and configure these as my seeds ?

Nope, see above.

[bbailey (Bill Bailey)]

Thanks Mark

Would you advise me to set the seed nodes in the cross-cluster configuration as the master nodes (as it is now) or the data nodes ? or all nodes ?

[warkolm (Mark Walkom)]

I'd use the data nodes, because they are the ones best used for queries (unless you had coordinating nodes)

[bbailey (Bill Bailey)]

That was what I meant in my "question 2" above but I guess I have the wrong terminology.

So I should either use the existing data nodes in each cluster as seeds or add at least one coordinating node to each cluster and use these as the seeds

You've been a great help, thanks

[warkolm (Mark Walkom)]

@bbailey

So I should either use the existing data nodes in each cluster as seeds or add at least one coordinating node to each cluster and use these as the seeds

Yep!

[alexus]

Using Kibana in a Production Environment | Kibana User Guide [6.1] | Elastic

the ideas is to run coordinate only elasticsearch node and kibana would access your cluster through that node.

[bbailey (Bill Bailey)]

It seems that if I run Kibana on a coordinating node in one cluster that I am then unable to search across both the local indices in that cluster AND remote nodes

How to run cross cluster search across local and remote indices

I am trying to setup my environment so that I can switch from using tribe nodes to use cross cluster search and running in to a few problems After some useful feedback in this forum I have configured my system with kibana pointing to a coordinating node in cluster_1 with cross cluster search configured so I can also search indices in cluster_2 and cluster_3. I can configure index patterns in Kibana to search for logs as follows:- logstash-* returns all local logs in clu…

This would suggest that the only way to search across multiple clusters would be to run kibana on a completely separate elasticsearch node so that I can use a wildcard in my index pattern to read data from all clusters.

( I have to point out though that I am using version 5.6.2 as there are limitations in my configuration that may prevent me from moving to 6.x yet )