Sorry if this has already been asked; I've mostly seen questions of how to deal with duplicate documents in the result set, but not how to actually locate and remove them from the index. We have a type within an index that contains ~7 million documents. Because this data was migrated from an earli…

Find and delete duplicate documents

PatrickKik (Patrick Kik) August 3, 2015, 5:05am 2

There is no straightforward solution.

You could scroll over all documents and query for each and every document id if there are any duplicates.

Topic		Replies	Views
Delete duplicate docs in ES 1.7 Elasticsearch	3	1550	July 5, 2017
Duplicate documents in Elasticsearch Elasticsearch	1	1002	June 23, 2017
Elasticsearch Head - Find and Remove Duplicate Documents Elasticsearch	1	470	December 31, 2020
Delete all docs that have duplicate field values Elasticsearch	5	420	March 10, 2022
Duplicate Deletion in Elasticsearch 2.X Elasticsearch	2	590	July 25, 2017