# How to retain only one record from duplicate records?

**URL:** <https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178>\
**Category:** Elasticsearch\
**Created:** [June 11, 2019, 11:54am UTC](https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178 "2019-06-11T11:54:19Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sana1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sana1/32/46439_2.png) [@sana1](https://discuss.elastic.co/u/sana1)\
**Post date:** [June 11, 2019, 11:54am UTC](https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178/1 "2019-06-11T11:54:20Z")

</div>

I have a database with duplicate results, i need to delete all other results and retain one.  
I have the query to find duplicate records  
POST /data\_12/\_search  
{  
"size": 0,  
"aggs": {  
"duplicateCount": {  
"terms": {  
"field": "col2.keyword",  
"min\_doc\_count": 2,  
"size": 10  
},  
"aggs": {  
"duplicateDocuments": {  
"top\_hits": {  
"size": 100  
}  
}  
}  
}  
}  
}

Please help me with the delete query

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [June 11, 2019, 12:28pm UTC](https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178/2 "2019-06-11T12:28:03Z")

</div>

Could you make col2 your choice of ID for the elasticsearch document to avoid this situation in the first place?

---

<div class="post-metadata">

**Author:** ![sana1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sana1/32/46439_2.png) [@sana1](https://discuss.elastic.co/u/sana1)\
**Post date:** [June 12, 2019, 4:47am UTC](https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178/3 "2019-06-12T04:47:39Z")

</div>

I donot get your point @Mark_Harwood

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [June 12, 2019, 6:40am UTC](https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178/4 "2019-06-12T06:40:00Z")

</div>

> [@sana1](#):
>
> I donot get your point

Elasticsearch IDs are the equivalent of a primary key - it ensures there’s only one doc with that ID. If you don’t supply an ID (and many don’t) elasticsearch invents a new unique one. If you do supply an ID it checks to see if this already exists

---

<div class="post-metadata">

**Author:** ![sana1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sana1/32/46439_2.png) [@sana1](https://discuss.elastic.co/u/sana1)\
**Post date:** [June 12, 2019, 6:43am UTC](https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178/5 "2019-06-12T06:43:01Z")

</div>

Please tell me procedure or code? Should I change in the ES template or should I change in the query?  
If in the query then how to do it?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [June 12, 2019, 7:13am UTC](https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178/6 "2019-06-12T07:13:02Z")

</div>

It’s not the template or the query. It’s the client that does the write - as well as giving the JSON body you can also pass an ID in our write APIs

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 10, 2019, 7:13am UTC](https://discuss.elastic.co/t/how-to-retain-only-one-record-from-duplicate-records/185178/7 "2019-07-10T07:13:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.