# Elasticsearch input on logstash duplicating documents

**URL:** <https://discuss.elastic.co/t/elasticsearch-input-on-logstash-duplicating-documents/249544>\
**Category:** Logstash\
**Created:** [September 22, 2020, 3:10pm UTC](https://discuss.elastic.co/t/elasticsearch-input-on-logstash-duplicating-documents/249544 "2020-09-22T15:10:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Matthew\_Field](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_field/32/26599_2.png) [@Matthew\_Field](https://discuss.elastic.co/u/Matthew_Field)\
**Post date:** [September 22, 2020, 3:10pm UTC](https://discuss.elastic.co/t/elasticsearch-input-on-logstash-duplicating-documents/249544/1 "2020-09-22T15:10:25Z")

</div>

According to the docs, if i don't set a schedule, the logstash input plugin should run once and only once. However i find logstash keeps adding new duplicate/triplicate etc documents to my output for the longer i run.

My config is:

```auto
        elasticsearch{
                hosts=>"https://10.x.x.x:9200"
                ca_file=>"/etc/logstash/ca.crt"
                user=>"elastic"
                password=>"password"
                index=>"my-index*"
                query=> '{ "query": { "match_all": {} } }'
                size => 500
                scroll => "5m"
                docinfo => true
}

```

I know that i could add the ID to the output config so that the documents dont get duplicated, but i would then be putting lots of unnecessary load on the output.

Shouldnt logstash just stop the pipeline once the initial document set has gone through the pipeline? (the query should be run once and only once)  
Is there something else wrong here?  
Where should the scroll ID be getting stored?  
Thanks in advance for any help you can give.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 20, 2020, 3:10pm UTC](https://discuss.elastic.co/t/elasticsearch-input-on-logstash-duplicating-documents/249544/2 "2020-10-20T15:10:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.