# Removing Duplicate documents in ElasticSearch

**URL:** <https://discuss.elastic.co/t/removing-duplicate-documents-in-elasticsearch/180876>\
**Category:** Elasticsearch\
**Created:** [May 13, 2019, 7:55pm UTC](https://discuss.elastic.co/t/removing-duplicate-documents-in-elasticsearch/180876 "2019-05-13T19:55:47Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Praneet\_Khandelwal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/praneet_khandelwal/32/40565_2.png) [@Praneet\_Khandelwal](https://discuss.elastic.co/u/Praneet_Khandelwal)\
**Post date:** [May 13, 2019, 7:55pm UTC](https://discuss.elastic.co/t/removing-duplicate-documents-in-elasticsearch/180876/1 "2019-05-13T19:55:47Z")

</div>

I have documents coming in from Logstash to Elasticsearch. I would like the following functionality. whenever a new document comes, it gets indexed in ES only if it is already not present in ES. As of now, I was trying to use fingerprint plugin of logstash, but it updates/ overwrites the original document. But I would like if the new document gets dropped. Is there a way to do this?

---

<div class="post-metadata">

**Author:** ![martinr\_ubi](https://avatars.discourse-cdn.com/v4/letter/m/b5e925/32.png) [@martinr\_ubi](https://discuss.elastic.co/u/martinr_ubi)\
**Post date:** [May 14, 2019, 12:23am UTC](https://discuss.elastic.co/t/removing-duplicate-documents-in-elasticsearch/180876/2 "2019-05-14T00:23:45Z")

</div>

Hi,

You need to change the op\_type from "index" to "create". Look here:  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#docs-bulk)  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index\_.html#operation-type](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#operation-type)  
[https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-action](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-action)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 11, 2019, 12:23am UTC](https://discuss.elastic.co/t/removing-duplicate-documents-in-elasticsearch/180876/3 "2019-06-11T00:23:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.