# Wait for file input plugin 90 seconds

**URL:** <https://discuss.elastic.co/t/wait-for-file-input-plugin-90-seconds/217627>\
**Category:** Logstash\
**Created:** [February 3, 2020, 1:16pm UTC](https://discuss.elastic.co/t/wait-for-file-input-plugin-90-seconds/217627 "2020-02-03T13:16:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\
**Post date:** [February 3, 2020, 1:16pm UTC](https://discuss.elastic.co/t/wait-for-file-input-plugin-90-seconds/217627/1 "2020-02-03T13:16:29Z")

</div>

I'm using input plugin to load CSV files into ES. Since files are created on another machine and afterward copied to local filesystem, I need to wait for file to completely be copied. We have issues with reading files (if the copy process is not finished), probably because of NFS copy - transfer stuff.

Want to create pipeline that will wait after file is created to wait 90 seconds before it starts reading it (90 seconds should be enough to complete the transfer). This is my input part of pipeline:

```
input{
    file{
        id => "import_files"
        path => "/logstash/files/csv/history_oracle_*"
        start_position => "beginning"
        sincedb_path => "/etc/logstash/sincedb_files"
        sincedb_clean_after => 4
        file_completed_action => "log"
        file_completed_log_path => "/etc/logstash/completed_files"
        stat_interval => "1000 ms"
        discover_interval => 90
    }
}

```

As far as I get it, `discover_interval * stat_interval => 90 seconds`, but I get errors again in the almost same time like time when file was created. How can I force logstash to wait after it sees the file to actually read from it in N seconds.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 3, 2020, 4:42pm UTC](https://discuss.elastic.co/t/wait-for-file-input-plugin-90-seconds/217627/2 "2020-02-03T16:42:51Z")

</div>

> [@ansamHox](#):
>
> How can I force logstash to wait after it sees the file to actually read from it in N seconds.

I don't think you can.

---

<div class="post-metadata">

**Author:** ![ansamHox](https://avatars.discourse-cdn.com/v4/letter/a/54ee81/32.png) [@ansamHox](https://discuss.elastic.co/u/ansamHox)\
**Post date:** [February 11, 2020, 10:17am UTC](https://discuss.elastic.co/t/wait-for-file-input-plugin-90-seconds/217627/3 "2020-02-11T10:17:28Z")

</div>

Great 🙂

Any suggestion how to delay this NFS copying errors, what else could I do, I'm really out of ideas because whatever I do, same error occurs 😕

---

<div class="post-metadata">

**Author:** ![ITIC](https://avatars.discourse-cdn.com/v4/letter/i/90ced4/32.png) [@ITIC](https://discuss.elastic.co/u/ITIC)\
**Post date:** [February 11, 2020, 10:27am UTC](https://discuss.elastic.co/t/wait-for-file-input-plugin-90-seconds/217627/4 "2020-02-11T10:27:46Z")

</div>

Hi

Just a crazy thought. Maybe you could do the NFS transfer to a temp file, and when that transfer is finished you copy (or mv) the temp file into the real file. This second copy would be local and hence very fast, hopefully fast enough for your `input{}` plugin to work as expected.

Hope this helps.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 10, 2020, 10:27am UTC](https://discuss.elastic.co/t/wait-for-file-input-plugin-90-seconds/217627/5 "2020-03-10T10:27:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.