# Delete all docs that have duplicate field values

**URL:** <https://discuss.elastic.co/t/delete-all-docs-that-have-duplicate-field-values/296702>\
**Category:** Elasticsearch\
**Created:** [February 9, 2022, 9:59am UTC](https://discuss.elastic.co/t/delete-all-docs-that-have-duplicate-field-values/296702 "2022-02-09T09:59:08Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)\
**Post date:** [February 9, 2022, 9:59am UTC](https://discuss.elastic.co/t/delete-all-docs-that-have-duplicate-field-values/296702/1 "2022-02-09T09:59:08Z")

</div>

Hello, I ingest some data into my cluster, and turned out some of my data has been ingested twice, so some of the data is doubled.  
My index has unique field, because some of it is doubled, I'm trying to search for duplicate field by using the terms aggregation, here's my command

```auto
GET <index>/_search
{
"size": 10000,
    "aggs": {
        "duplicateNames": {
            "terms": {
                "field": "EmployeeName",
                "min_doc_count": 2
            }
        }
    }
}

```

from that command, I can find the duplicate value. But is there any way to delete just the "duplicate" value and skip the original one?  
tldr can I delete one from two same docs value?

Any help is appreciated, Thanks

---

<div class="post-metadata">

**Author:** ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)\
**Post date:** [February 9, 2022, 6:22pm UTC](https://discuss.elastic.co/t/delete-all-docs-that-have-duplicate-field-values/296702/2 "2022-02-09T18:22:53Z")

</div>

anyone got any clue about this?

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [February 9, 2022, 6:49pm UTC](https://discuss.elastic.co/t/delete-all-docs-that-have-duplicate-field-values/296702/3 "2022-02-09T18:49:14Z")

</div>

Have a look to this post

> **[How to Find and Remove Duplicate Documents in Elasticsearch](https://www.elastic.co/blog/how-to-find-and-remove-duplicate-documents-in-elasticsearch)**
>
> Learn how to detect and remove duplicate documents from Elasticsearch using Logstash or a custom Python script.

---

<div class="post-metadata">

**Author:** ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)\
**Post date:** [February 10, 2022, 1:47am UTC](https://discuss.elastic.co/t/delete-all-docs-that-have-duplicate-field-values/296702/4 "2022-02-10T01:47:37Z")

</div>

hi, thanks for the answer,  
I will look into this asap, will let you know after I try this method

---

<div class="post-metadata">

**Author:** ![alfianaf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alfianaf/32/93192_2.png) [@alfianaf](https://discuss.elastic.co/u/alfianaf)\
**Post date:** [February 10, 2022, 10:48am UTC](https://discuss.elastic.co/t/delete-all-docs-that-have-duplicate-field-values/296702/5 "2022-02-10T10:48:57Z")

</div>

thanks for the help, finally I can do the thing, appreciate it

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 10, 2022, 10:49am UTC](https://discuss.elastic.co/t/delete-all-docs-that-have-duplicate-field-values/296702/6 "2022-03-10T10:49:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.