# Delete duplicate docs in ES 1.7 **URL:** https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808 **Category:** Elasticsearch **Created:** [September 18, 2016, 12:38pm UTC](https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808 "2016-09-18T12:38:11Z") **Posts on this page:** 1 **Showing post:** 2
### Author: ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec) #### Post date: [September 19, 2016, 12:17pm UTC](https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808/2 "2016-09-19T12:17:49Z")
How do you define a duplicate? A brief search across the history of the discuss forum I'm currently typing this answer in revealed the following threads which might be helpful for you: > [@How to filter out duplicate documents across multiple types?](https://discuss.elastic.co/t/how-to-filter-out-duplicate-documents-across-multiple-types/19636): > > We have certain documents stored across multiple types with translated values, for example, US and ES types has same document but with different values in title fields. Example: US: { "title":"Manning: Spring in Action, Third Edition" } ES: { "title":"Manning : Primavera en Acción , Tercera Edición" } So, when I search for "Manning" across all types, I only want one document. I can certainly remove the duplicates in my code, but then I can not use pagination. Any one know how to r… > [@Filebeat, Logstash, Elasticsearch robustness and duplicated documents](https://discuss.elastic.co/t/filebeat-logstash-elasticsearch-robustness-and-duplicated-documents/42711): > > For logging in AWS EC2, I'm testing the robustness of the chain Filebeat, Logstash, Elasticsearch. I have one AMI with an appplication + Filebeat, one with Logstash and one with Elastisearch + Kibana. With the application running I try to reboot one of these 3 machines and see what happens when it's back available. The good news is that I never loose any line of log. The less good one is that most of the times I end up having duplicated logs in Elasticsearch. Typically I generate X lines (let's… > [@Using elasticsearch to find duplicates in dataset](https://discuss.elastic.co/t/using-elasticsearch-to-find-duplicates-in-dataset/8483): > > Hello, I am currently evaluating elasticsearch for a very specific task which is removing duplicates from a contacts list, from my initial tests it looks like it would works but there are still some shadows I hope you can help me with. What I was planning to do is: load the data from some csv files normalize the fields (phone numbers, addresses) load the data into elasticsearch run a bunch of queries on the data to find/remove/merge the duplicates export the data back into csv The first … > [@Is there any way to remove duplicated search result in ES?](https://discuss.elastic.co/t/is-there-any-way-to-remove-duplicated-search-result-in-es/15317/4): > > Thank you for your rapid reply . it is true that i can custom my own search action, but i can not override the default search action .so, it is not what i want. at indexing time , there are serval listeners to install plugins, but at searching time there is hardly any listener to extend the search operation except the search action . why not provide a opportunity to install my own plugin to extend the search phase , because it seems to be simple from the source code . i should give up th… Hope this helps to get you started, Isabel --- _[View the full topic](https://discuss.elastic.co/t/delete-duplicate-docs-in-es-1-7/60808)._