# Check if similare documents exists?

**URL:** <https://discuss.elastic.co/t/check-if-similare-documents-exists/114432>\
**Category:** Elasticsearch\
**Created:** [January 7, 2018, 9:59pm UTC](https://discuss.elastic.co/t/check-if-similare-documents-exists/114432 "2018-01-07T21:59:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Antoine94](https://avatars.discourse-cdn.com/v4/letter/a/76d3ee/32.png) [@Antoine94](https://discuss.elastic.co/u/Antoine94)\
**Post date:** [January 7, 2018, 9:59pm UTC](https://discuss.elastic.co/t/check-if-similare-documents-exists/114432/1 "2018-01-07T21:59:46Z")

</div>

Hi,

I'm doing some tests with ELK (Latest version), here is my use case:

I'm indexing some documents containing phone activity, where I use these fields:

- SYSTEMA.EndUserName
- SYSTEMA.EndUserPhoneNumber
- SYSTEMA.CounterpartPhoneNumber
- SYSTEMA.Call Direction
- SYSTEMA.Date

On other side, I'm also indexing similar documents

- SYSTEMB.EndUserName
- SYSTEMB.EndUserPhoneNumber
- SYSTEMB.CounterpartPhoneNumber
- SYSTEMB.Call Direction
- SYSTEMB.Date

My target is to search for any SYSTEMA document if a similar SYSTEMB document exists (With same EndUserName, EndUserPhoneNumber, CounterpartPhoneNumber, CallDirection) and approximatively same date (With few second of difference).

I thought about creating a scripted field for SYSTEMA checking the existence of a similar SYSTEMB document, what do you think?

Do you have another way to achieve this?

Many thanks,  
Regards.

---

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [January 8, 2018, 1:15am UTC](https://discuss.elastic.co/t/check-if-similare-documents-exists/114432/2 "2018-01-08T01:15:20Z")

</div>

Why not have the fields all be on the parent level and add an extra called "system" and then the value is "a" or "b"? Then just have a cardinality aggregation sorted by lowest to highest. That way, it will collect unique values into buckets and then display the buckets that have the most duplicate documents for those fields.

---

<div class="post-metadata">

**Author:** ![Antoine94](https://avatars.discourse-cdn.com/v4/letter/a/76d3ee/32.png) [@Antoine94](https://discuss.elastic.co/u/Antoine94)\
**Post date:** [January 8, 2018, 9:04am UTC](https://discuss.elastic.co/t/check-if-similare-documents-exists/114432/3 "2018-01-08T09:04:30Z")

</div>

Thank you for your reply.

I'm looking for the most effective way to achieve this, do you think that adding a scripted field looking for a similar document is the best way?

I'm thinking about something like a scripted field for the system A telling if a similar document for system B is existing.

What do you think?

Thanks again for your answer.

---

<div class="post-metadata">

**Author:** ![arisbanach](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@arisbanach](https://discuss.elastic.co/u/arisbanach)\
**Post date:** [January 8, 2018, 2:20pm UTC](https://discuss.elastic.co/t/check-if-similare-documents-exists/114432/4 "2018-01-08T14:20:25Z")

</div>

I'm not sure since I'm relatively new to Elasticsearch, sorry! I think that a scripted field could work, but I don't know if there are better ways.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 5, 2018, 2:20pm UTC](https://discuss.elastic.co/t/check-if-similare-documents-exists/114432/5 "2018-02-05T14:20:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.