# Delete duplicate docs in ES 1.7 **URL:** \ **Category:** Elasticsearch\ **Created:** [September 18, 2016, 12:38pm UTC](https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808 "2016-09-18T12:38:11Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![muhamadli302](https://avatars.discourse-cdn.com/v4/letter/m/b4bc9f/32.png) [@muhamadli302](https://discuss.elastic.co/u/muhamadli302)\ **Post date:** [September 18, 2016, 12:38pm UTC](https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808/1 "2016-09-18T12:38:11Z")
Hello, I'm having a problem finding the right query for searching and deleting duplicated documents in my index. I'm using ES 1.7. Thank you for your help 🙂 ---
**Author:** ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)\ **Post date:** [September 19, 2016, 12:17pm UTC](https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808/2 "2016-09-19T12:17:49Z")
How do you define a duplicate? A brief search across the history of the discuss forum I'm currently typing this answer in revealed the following threads which might be helpful for you: > [@How to filter out duplicate documents across multiple types?](https://discuss.elastic.co/t/how-to-filter-out-duplicate-documents-across-multiple-types/19636): > > We have certain documents stored across multiple types with translated values, for example, US and ES types has same document but with different values in title fields. Example: US: { "title":"Manning: Spring in Action, Third Edition" } ES: { "title":"Manning : Primavera en Acción , Tercera Edición" } So, when I search for "Manning" across all types, I only want one document. I can certainly remove the duplicates in my code, but then I can not use pagination. Any one know how to r… > [@Filebeat, Logstash, Elasticsearch robustness and duplicated documents](https://discuss.elastic.co/t/filebeat-logstash-elasticsearch-robustness-and-duplicated-documents/42711): > > For logging in AWS EC2, I'm testing the robustness of the chain Filebeat, Logstash, Elasticsearch. I have one AMI with an appplication + Filebeat, one with Logstash and one with Elastisearch + Kibana. With the application running I try to reboot one of these 3 machines and see what happens when it's back available. The good news is that I never loose any line of log. The less good one is that most of the times I end up having duplicated logs in Elasticsearch. Typically I generate X lines (let's… > [@Using elasticsearch to find duplicates in dataset](https://discuss.elastic.co/t/using-elasticsearch-to-find-duplicates-in-dataset/8483): > > Hello, I am currently evaluating elasticsearch for a very specific task which is removing duplicates from a contacts list, from my initial tests it looks like it would works but there are still some shadows I hope you can help me with. What I was planning to do is: load the data from some csv files normalize the fields (phone numbers, addresses) load the data into elasticsearch run a bunch of queries on the data to find/remove/merge the duplicates export the data back into csv The first … > [@Is there any way to remove duplicated search result in ES?](https://discuss.elastic.co/t/is-there-any-way-to-remove-duplicated-search-result-in-es/15317/4): > > Thank you for your rapid reply . it is true that i can custom my own search action, but i can not override the default search action .so, it is not what i want. at indexing time , there are serval listeners to install plugins, but at searching time there is hardly any listener to extend the search operation except the search action . why not provide a opportunity to install my own plugin to extend the search phase , because it seems to be simple from the source code . i should give up th… Hope this helps to get you started, Isabel ---
**Author:** ![muhamadli302](https://avatars.discourse-cdn.com/v4/letter/m/b4bc9f/32.png) [@muhamadli302](https://discuss.elastic.co/u/muhamadli302)\ **Post date:** [September 22, 2016, 8:15am UTC](https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808/3 "2016-09-22T08:15:45Z")
Hi, Thank you for your replay, my definition for a duplicate is the same document with the same unique field under the same index. This situation is caused by problems in inserting the data. So now I need to find an efficient way to find all the duplicates and delete them (leaving only one copy). It would be best if you could help us create a query that will find those duplicates and delete them. Thank you! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:18pm UTC](https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808/4 "2017-07-05T22:18:15Z")