Compare fields of different documents in the same index

Elastic Stack Kibana
1

There are a lot of discussions around this topic but haven't found a solution for my problem.
I am trying to compare fields of different documents in the same index. For example,let's say i have three documents as below :
Index1
Doc 1
Field1 = Value1
Field2 = Value2
Doc 2
field1 = Value1
field2 = Value_2
Doc 3
field1 = Value_1
field2 = Value_2

I want to display all the documents where Field1.value == field1.value (according to the example, it should display Doc1 and Doc2). I don't know what the value will be, so I can't make a query by value and also I read that scripted fields can't be used here since it works only within a document.
Documents are bulk imported from two different sources, the comparison can't be done on the client's side.

I really appreciate any help since I am struggling with this for a long time now. Thanks in advance.

2

It sounds like you want to explore individual documents with filters that you don't know ahead of time. Can you add filters in Discover? For example, Field1.value: "Value1" is a query you could run. If you want to provide a nicer UI for this, you could save your Discover search and embed it into a dashboard, and then add an Input Controls widget.

3

Thank you Wylie.
But let me try to explain my problem more clearly. Below is a simplified structure of my docs.
Index1
Doc 1 (ingested by source1)
Field1 = "ABC"
Field2 = "DEF"
Doc 2 (ingested by source2)
field1 = "ABC"
field2 = "EDF"
Doc 3 (ingested by source1)
Field1 = "GHI"
Field2 = "JKL"
Doc 4 (ingested by source2)
field1 = "GHI"
field2 = "JL"
Now, I want to loop through all the different field1.value values(docs ingested by source2) and find all the docs (ingested by source1) if field1.value==Field1.value.
Is this even possible in kibana?

4

I don't really understand this use case, but because it needs to run in two passes you have limited options:

  1. The only client-side way to run two passes in Kibana is using Canvas. You can make a request, and then script something to make another request
  2. In a batch process, you can set a field like doc.hasMatching: true in documents and then use this in Kibana
  3. You might be able to do this using a Continuous Transform
5

Thank you and I will look into your options

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.