# Remove File After Read

**URL:** <https://discuss.elastic.co/t/remove-file-after-read/162809>\
**Category:** Logstash\
**Created:** [January 3, 2019, 12:57pm UTC](https://discuss.elastic.co/t/remove-file-after-read/162809 "2019-01-03T12:57:05Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\
**Post date:** [January 3, 2019, 12:57pm UTC](https://discuss.elastic.co/t/remove-file-after-read/162809/1 "2019-01-03T12:57:06Z")

</div>

Hey I have a quick question. I was wondering if there is anyway to remove a file after it has been read by logstash. I have a feeling mine keeps hanging on the same file over and over but when I check the query there is only 1 entry so thats a plus. But I dont want it to keep reading the same file over and over and over.

I have roughly 400 .csv files that range from 400 megs to as small as 100 megs.

I have provided a copy of my config file I am using.

```
input {
    file {
    path => "/home/dtengine/Documents/conversion/*.*"
    start_position => "beginning"
    sincedb_path => "dev/null"
    }
}
    filter {
    csv {
    separator => ","
    }
}
output {
    elasticsearch { host => ["http://localhost:9200"] }
    index => "some_name"
}
stdout {codec => rubydebug}
}
```

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [January 3, 2019, 2:29pm UTC](https://discuss.elastic.co/t/remove-file-after-read/162809/2 "2019-01-03T14:29:51Z")

</div>

> [@OpSecMonkey](#):
>
> ```auto
> sincedb_path => "dev/null"
> 
> ```

The path to your system's null device is `/dev/null` (that leading slash is important). My guess is that with your current configuration the File Input Plugin is creating a sincedb and placing it at the path `dev/null` _relative to_ your config directory; if this is the case, it could explain why the input is "remembering" where it left off and refusing to reprocess files after a restart.

* * *

If you do need to delete files after they have been processed, the [File Input Plugin's `file_completed_action` directive](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-file_completed_action) may be what you are looking for.

---

<div class="post-metadata">

**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\
**Post date:** [January 3, 2019, 4:29pm UTC](https://discuss.elastic.co/t/remove-file-after-read/162809/3 "2019-01-03T16:29:07Z")

</div>

Got it. I will change it and add the `/` thanks for noticing that. I will go ahead and stop my process and fix it. Its been on one CSV thats about 40 gigs that I had added for about 12 hrs.

---

<div class="post-metadata">

**Author:** ![OpSecMonkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/opsecmonkey/32/46110_2.png) [@OpSecMonkey](https://discuss.elastic.co/u/OpSecMonkey)\
**Post date:** [January 3, 2019, 7:26pm UTC](https://discuss.elastic.co/t/remove-file-after-read/162809/4 "2019-01-03T19:26:23Z")

</div>

How would I add `file_completed_action` & `'file_sort_direction'`

Would it be like

```
input {
    file {
    path => "/home/dtengine/Documents/conversion/*.*"
    start_position => "beginning"
    file_sort_direction => "asc"
    file_completed_action => "delete"
    sincedb_path => "/dev/null"
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 31, 2019, 7:26pm UTC](https://discuss.elastic.co/t/remove-file-after-read/162809/5 "2019-01-31T19:26:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.