# Wtacher or Ingest pipeline avoiding duplicates

**URL:** <https://discuss.elastic.co/t/wtacher-or-ingest-pipeline-avoiding-duplicates/345747>\
**Category:** Elasticsearch\
**Created:** [October 25, 2023, 3:36pm UTC](https://discuss.elastic.co/t/wtacher-or-ingest-pipeline-avoiding-duplicates/345747 "2023-10-25T15:36:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![elk-user-0001](https://avatars.discourse-cdn.com/v4/letter/e/e47774/32.png) [@elk-user-0001](https://discuss.elastic.co/u/elk-user-0001)\
**Post date:** [October 25, 2023, 3:36pm UTC](https://discuss.elastic.co/t/wtacher-or-ingest-pipeline-avoiding-duplicates/345747/1 "2023-10-25T15:36:35Z")

</div>

Hello colleagues!

I have a question.

Is there a way to put in watcher or ingest pipelines an update based on a document\_id ( field, fields, fingerprint... ) to avoid duplicates as in the logstash output ?

I have a watcher that is doing some aggregations by date ranges and we want to avoid that these duplicates can be produced.

Thanks in advance !  
Best regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 22, 2023, 3:36pm UTC](https://discuss.elastic.co/t/wtacher-or-ingest-pipeline-avoiding-duplicates/345747/2 "2023-11-22T15:36:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.