# How to find duplicate numbers in multiple fields?
**URL:** https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346
**Category:** Elasticsearch
**Created:** [August 8, 2019, 2:50am UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346 "2019-08-08T02:50:30Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Smkumaran](https://discuss.elastic.co/u/Smkumaran)
#### Post date: [August 8, 2019, 2:50am UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346/1 "2019-08-08T02:50:30Z")
Hello,
Can anyone help me on this please?.
I need to match the same values with two different index's and 4 fields in total and then get the results if all four only matches.
Example :
Index-1 : log-1
Fields : causerid , casessionid
Index-2: log2
Fields: caloginid , caexpireid
If all 4 matches the same value=1234 , then it has to populate the report.
Much appreciated your precious help.
---
### Author:  [@gabriel\_tessier](https://discuss.elastic.co/u/gabriel_tessier)
#### Post date: [August 9, 2019, 4:01am UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346/2 "2019-08-09T04:01:28Z")
Hi @Smkumaran,
Just one question to remove misunderstood, do you know about the value 1234 before search or not?
I mean you search all the doc with causerid=1234 and casesessionid=1234 and caloginid=1234 and caexpired=1234
OR
you have all your log and you want to make a join like in SQL?
if so one solution can be to list all unique causerid (for example) and loop for each value to search the three others values. May use mget or msearch. This solution can be used only if you have few unique ids.
After it depends on your data and what you expect as performance.
---
### Author:  [@Smkumaran](https://discuss.elastic.co/u/Smkumaran)
#### Post date: [August 11, 2019, 4:50pm UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346/3 "2019-08-11T16:50:14Z")
Hi @gabriel_tessier
Thanks for your help. The value is not unique or known. Means, it could be anything if that matches, then expecting lists. Based on the this , I need to pull out other report.
I'm very new to ELK. If you can send me the complete syntax, would be very helpful.
Thanks in advance.
---
### Author:  [@Smkumaran](https://discuss.elastic.co/u/Smkumaran)
#### Post date: [August 12, 2019, 4:23pm UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346/5 "2019-08-12T16:23:52Z")
Hello,
Anyone help on this pls?
---
### Author:  [@Smkumaran](https://discuss.elastic.co/u/Smkumaran)
#### Post date: [August 13, 2019, 3:33am UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346/6 "2019-08-13T03:33:49Z")
Hello,
Anyone help on this pls ?
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [August 13, 2019, 4:14am UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346/7 "2019-08-13T04:14:19Z")
I don't think you can do it as you can't join 2 indices with elasticsearch.
You can run a manual job which scans every single document from index 1 and for each run a job in index 2 to search for the terms found in index 1.
It will be slow for sure.
---
### Author:  [@gabriel\_tessier](https://discuss.elastic.co/u/gabriel_tessier)
#### Post date: [August 22, 2019, 4:04am UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346/8 "2019-08-22T04:04:01Z")
Hi,
Found this solution that can also help depend on your needs and data:
> [@How to merge two indexes based on common field in Elasticsearch?](https://discuss.elastic.co/t/how-to-merge-two-indexes-based-on-common-field-in-elasticsearch/194114/3):
>
> You could use the scroll api to do a single search across both indices sorted by customer-Id. That would give you a single stream of results with related docs next to each other in the results order. A custom script could consume this stream and merge the doc pairs and then add the merged doc to a new index using the bulk api
if you merge your 2 indices in one and remove the duplicate, you can use this third index to populate your report.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 19, 2019, 4:04am UTC](https://discuss.elastic.co/t/how-to-find-duplicate-numbers-in-multiple-fields/194346/9 "2019-09-19T04:04:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.