# How to calculate unique contact duplicates using different fields

**URL:** <https://discuss.elastic.co/t/how-to-calculate-unique-contact-duplicates-using-different-fields/370716>\
**Category:** Elasticsearch\
**Tags:** aggregations\
**Created:** [November 18, 2024, 7:24pm UTC](https://discuss.elastic.co/t/how-to-calculate-unique-contact-duplicates-using-different-fields/370716 "2024-11-18T19:24:52Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![andreyshiryaev13](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreyshiryaev13/32/122198_2.png) [@andreyshiryaev13](https://discuss.elastic.co/u/andreyshiryaev13)\
**Post date:** [November 18, 2024, 7:24pm UTC](https://discuss.elastic.co/t/how-to-calculate-unique-contact-duplicates-using-different-fields/370716/1 "2024-11-18T19:24:52Z")

</div>

Hi, I am trying to find an effective way to count unique contacts. Elastic 8.15  
Index structure

```auto
{
id, email, phone, first, last
}

```

Data Example

```auto
{
1, user@gmail.com, 1111111, Tom, Hanks
},
{
2, user2@gmail.com, 1111111, Tom, Hanks
},
{
3, user@gmail.com, 22222, Tom, Hanks
},
{
4, user4@gmail.com, 22222, Kate, Ragan
},

```

Each record contains something common and there are here 4 potential duplicate records

when I count using each field I receive  
email - 2  
phone - 2,  
FullName -3  
The expected result is 4

phone or email or first + last - could determine contact

In what ways could I use to calculate that? I will be great for any ideas  
Thanks

---

<div class="post-metadata">

**Author:** ![RabBit\_BR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rabbit_br/32/82261_2.png) [@RabBit\_BR](https://discuss.elastic.co/u/RabBit_BR)\
**Post date:** [November 19, 2024, 7:01pm UTC](https://discuss.elastic.co/t/how-to-calculate-unique-contact-duplicates-using-different-fields/370716/2 "2024-11-19T19:01:56Z")

</div>

Hi @andreyshiryaev13

You can use a new field to create a unique identifier using properties of the document. Below, I used a processor to create fields that can be used as a unique key. I hope this helps as a starting point.

```auto
POST /_ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "set": {
          "field": "_source.unique_id_email_full_name",
          "value": "{{_source.email}}-{{_source.first}}-{{_source.last}}"
        }
      },
      {
        "set": {
          "field": "_source.unique_id_phone_full_name",
          "value": "{{_source.phone}}-{{_source.first}}-{{_source.last}}"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "1",
      "_source": {
        "email": "user@gmail.com",
        "phone": "1111111",
        "first": "Tom",
        "last": "Hanks"
      }
    },
    {
      "_index": "index",
      "_id": "2",
      "_source": {
        "email": "user2@gmail.com",
        "phone": "1111111",
        "first": "Tom",
        "last": "Hanks"
      }
    },
    {
      "_index": "index",
      "_id": "3",
      "_source": {
        "email": "user@gmail.com",
        "phone": "22222",
        "first": "Tom",
        "last": "Hanks"
      }
    },
    {
      "_index": "index",
      "_id": "4",
      "_source": {
        "email": "user4@gmail.com",
        "phone": "22222",
        "first": "Kate",
        "last": "Ragan"
      }
    }
  ]
}

```

---

<div class="post-metadata">

**Author:** ![andreyshiryaev13](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andreyshiryaev13/32/122198_2.png) [@andreyshiryaev13](https://discuss.elastic.co/u/andreyshiryaev13)\
**Post date:** [November 19, 2024, 9:00pm UTC](https://discuss.elastic.co/t/how-to-calculate-unique-contact-duplicates-using-different-fields/370716/3 "2024-11-19T21:00:04Z")

</div>

thank you for your answer i will try to use that

it looks like a multi-term aggregation but I need another not all 3 fields.

I have 3 criteria(phone or email or name) and each of them can determine a duplicate row.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 17, 2024, 9:00pm UTC](https://discuss.elastic.co/t/how-to-calculate-unique-contact-duplicates-using-different-fields/370716/4 "2024-12-17T21:00:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.