How to find duplicate numbers in multiple fields?

Elastic Stack Elasticsearch
1

Hello,

Can anyone help me on this please?.

I need to match the same values with two different index's and 4 fields in total and then get the results if all four only matches.

Example :

Index-1 : log-1
Fields : causerid , casessionid

Index-2: log2
Fields: caloginid , caexpireid

If all 4 matches the same value=1234 , then it has to populate the report.

Much appreciated your precious help.

2

Hi @Smkumaran,

Just one question to remove misunderstood, do you know about the value 1234 before search or not?

I mean you search all the doc with causerid=1234 and casesessionid=1234 and caloginid=1234 and caexpired=1234

OR

you have all your log and you want to make a join like in SQL?

if so one solution can be to list all unique causerid (for example) and loop for each value to search the three others values. May use mget or msearch. This solution can be used only if you have few unique ids.
After it depends on your data and what you expect as performance.

3

Hi @gabriel_tessier

Thanks for your help. The value is not unique or known. Means, it could be anything if that matches, then expecting lists. Based on the this , I need to pull out other report.

I'm very new to ELK. If you can send me the complete syntax, would be very helpful.

Thanks in advance.

5

Hello,

Anyone help on this pls?

6

Hello,

Anyone help on this pls ?

7

I don't think you can do it as you can't join 2 indices with elasticsearch.
You can run a manual job which scans every single document from index 1 and for each run a job in index 2 to search for the terms found in index 1.
It will be slow for sure.

8

Hi,

Found this solution that can also help depend on your needs and data:

if you merge your 2 indices in one and remove the duplicate, you can use this third index to populate your report.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.