# How to identiry duplicates and delete it in index **URL:** \ **Category:** Elasticsearch\ **Created:** [June 22, 2022, 5:13am UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830 "2022-06-22T05:13:23Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Naga\_Prudhvi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/naga_prudhvi/32/99501_2.png) [@Naga\_Prudhvi](https://discuss.elastic.co/u/Naga_Prudhvi)\ **Post date:** [June 22, 2022, 5:13am UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830/1 "2022-06-22T05:13:23Z")
we are using elasticsearch 7.11.1 recently we observed an issue and below are the points for it. 1. we store data for every 15 mins interval and we get time stamp from our input file (ex: 05:00, 23:15, 20:30, 11:45 ) 2. recently we observed our input file at 23:15 has 1890 records, but index has 3533 records. 3. now we want to delete 1643 duplictae records from index,with out disturbing 1890 records. We need API query for that. for example input file name product sale id sai pen 100 1 kumar car 30 2 sai pen 100 1 sai pen 100 1 ram bike 288 3 kumar car 30 2 After deleting duplicates my index should loook like below, name product sale id sai pen 100 1 ram bike 288 3 kumar car 30 2 ---
**Author:** ![Naga\_Prudhvi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/naga_prudhvi/32/99501_2.png) [@Naga\_Prudhvi](https://discuss.elastic.co/u/Naga_Prudhvi)\ **Post date:** [June 22, 2022, 5:17am UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830/2 "2022-06-22T05:17:32Z")
I need help with 1. query to find only duplicates at 23:15 2. query to delete duplicates ---
**Author:** ![RabBit\_BR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rabbit_br/32/82261_2.png) [@RabBit\_BR](https://discuss.elastic.co/u/RabBit_BR)\ **Post date:** [June 22, 2022, 1:40pm UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830/3 "2022-06-22T13:40:12Z")
This [post](https://www.elastic.co/blog/how-to-find-and-remove-duplicate-documents-in-elasticsearch) is not new but it can give you a direction on how to perform the operation. ---
**Author:** ![RabBit\_BR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rabbit_br/32/82261_2.png) [@RabBit\_BR](https://discuss.elastic.co/u/RabBit_BR)\ **Post date:** [June 22, 2022, 1:42pm UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830/4 "2022-06-22T13:42:12Z")
It seems to be the same problem. > [@How to identify and remove duplicates in Elasticsearch index](https://discuss.elastic.co/t/how-to-identify-and-remove-duplicates-in-elasticsearch-index/307866): > > we are using elasticsearch 7.11.1 recently we observed an issue and below are the points for it. we store data for every 15 mins interval and we get time stamp from our input file (ex: 05:00, 23:15, 20:30, 11:45 ) recently we observed our input file at 23:15 has 1890 records, but index has 3533 records. now we want to delete 1643 duplicate records from index, with out disturbing 1890 records. We need API query for that. for example input file name product sale id sai pen 100 1 kumar car … ---
**Author:** ![Naga\_Prudhvi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/naga_prudhvi/32/99501_2.png) [@Naga\_Prudhvi](https://discuss.elastic.co/u/Naga_Prudhvi)\ **Post date:** [June 23, 2022, 3:46am UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830/5 "2022-06-23T03:46:05Z")
We are looking for an API query approch ---
**Author:** ![Naga\_Prudhvi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/naga_prudhvi/32/99501_2.png) [@Naga\_Prudhvi](https://discuss.elastic.co/u/Naga_Prudhvi)\ **Post date:** [June 23, 2022, 7:07am UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830/6 "2022-06-23T07:07:07Z")
we are looking for an API solution , and we didnt get that, so we are searching for the same, Much appreciated if get help with API query ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 23, 2022, 7:12am UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830/7 "2022-06-23T07:12:42Z")
As I pointed out in my response I do not believe there is a query that can be used with delete by query to do this (which seems to be what you are looking for). I would therefore recommend looking at the approaches described in the blog post I linked to. Most people frequenting the forums respond only if they have a solution, so if you do not receive any solution within a reasonable time period it is quite possible that what you are looking for is not possible. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 21, 2022, 7:13am UTC](https://discuss.elastic.co/t/how-to-identiry-duplicates-and-delete-it-in-index/307830/8 "2022-07-21T07:13:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.