Executing terms filter search on an index in Kibana

Elastic Stack Kibana
1

I want to filter execute terms filter search on my dashboard, but however I'm seeing no results, even though I could see that there is a match, and the string fields are indexed as not_analyzed, as suggested for terms filter search.

Details :
I have 2 indices - A & B.

Index A has the device logs which contains the list of IPs that were connected among other fields.
Index B has list of blacklisted IPs and their details.

I want to check if any of the blacklist IPs( in index B) are present in IP list in Index A.

Can you please let me know how to do this in Kibana.

2

I don't believe this type of visual join is easily possible in Kibana (or Elasticsearch for that matter) out of the box. See an old discussion from Kibana 3 days about it: https://github.com/elastic/kibana/issues/1012

You may need a custom client/UI to do that that accomplishes the join outside Kibana. I'm aware of 3rd party tools that try to add relational semantics on top of Elasticsearch, so that may be an option as well: https://siren.solutions/kibi/

3

Natarajan,

There might be a hacky workaround to what you want to do. It's not pretty, but should work.

If in index A and index B, you have a field for clientip which shares the same name. Then you could create a table visualization of the top N ips in index B (blacklist index) and a top N ips table for index A and place both on the dashboard. Then in the search box at the top, you could filter for ips from the blacklisted table (index B) and see if those ips show up in the other table (index A). But it would require that the field name for ips remain the same across indices.

4

Yep, I was following this method till now. But, it did'nt solve my problem. The count of blacklisted IPs are in 1000s and getting updated regulary. Hence the suggested method may not work out.

5

Thanks @tbragin . The Siren Kibi solves my problem, and they also support the latest version elasticsearch.

Are there any plans in pipeline to support those kind of features in Kibana ? I understand that this will require elasticsearch to support relational data models natively.

6

There are no such plans at this time, as it's hard to do just in the UI in a way that scales well to large data sets. We will continue to think about it though!

Note that stormpython had a good suggestion above for a way to make this work in Kibana, if IP field names match across indices.