How to run cross cluster search across local and remote indices


(Bill Bailey) #1

I am trying to setup my environment so that I can switch from using tribe nodes to use cross cluster search and running in to a few problems

After some useful feedback in this forum I have configured my system with kibana pointing to a coordinating node in cluster_1 with cross cluster search configured so I can also search indices in cluster_2 and cluster_3.

I can configure index patterns in Kibana to search for logs as follows:-

logstash-* returns all local logs in cluster_1 successfully
cluster_2:logstash-* returns all logs in remote cluster_2 successfully
cluster_3:logstash-* returns all logs in remote cluster_3 successfully
*:logstash-* returns all logs in remote cluster_2 and cluster_3 successfully

Question1. Is there a way to set up an index pattern to query all logs in all 3 clusters i.e. local and remote clusters ?

Question 2. Is there a way to search specific remote clusters ? I have tried the documented syntax of cluster_2,cluster_3:logstash-* and in Kibana 5.6.2 it does not work


Configuration of kibana with cross cluster search
(Tim Vernum) #2

Is there a way to set up an index pattern to query all logs in all 3 clusters i.e. local and remote clusters

There's 2 options, that may (or may not) suit you.

  1. logstash-*,*:logstash-* should do what you want. That will search locally and on all remote clusters.
  2. You can set up a cross-cluster prefix that points to the local cluster. So you can define cluster_1 with a seed of localhost:9300 then *:logstash-* would search all 3 clusters. It would use the cross-cluster-search mechanism, but cross cluster search is fairly low overhead so that shouldn't cause a problem.

I have tried the documented syntax of cluster_2,cluster_3:logstash-* and in Kibana 5.6.2 it does not work

Which document has that?
As far as Elasticsearch is concerned, that pattern is

  • the index named "cluster_2" in the local cluster
  • the indices named "logstash-*" in the "cluster_3" cluster

It's possible that Kibana has some feature that handles it differently, but once it gets to Elasticsearch we split the names by comma and then treat each part independently.


(Bill Bailey) #3

Regarding the incorrect Kibana documentation

It's in https://www.elastic.co/guide/en/kibana/current/management-cross-cluster-search.html


(Mark Walkom) #4

Thanks for pointing that out, I've raised https://github.com/elastic/kibana/pull/16751 to update it.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.