# Logstash file input not reparsing file

**URL:** https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854
**Category:** Logstash
**Created:** [December 13, 2019, 11:25pm UTC](https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854 "2019-12-13T23:25:54Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![konan\_pustolov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/konan_pustolov/32/59375_2.png) [@konan\_pustolov](https://discuss.elastic.co/u/konan_pustolov)
#### Post date: [December 13, 2019, 11:25pm UTC](https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854/1 "2019-12-13T23:25:55Z")

</div>

I have the following problem, I need logstash to reparse already parsed files:

Scenario that doesn't work but should:

- upload file to watched folder

- logstash processes it, saves to elastic, removes it (file\_completed\_action =\> "log\_and\_delete"), great

- I upload the same file again, same name, same content.

- logstash doesnt do anything, I want it to process it again

Here is my file input config:

```auto
file {
          mode => "read"
          exclude => "*.tif"
          path => ["/home/xmls/*.xml"]
          file_completed_action => "log_and_delete"
          file_completed_log_path => "/var/log/logstash/completed.log"
          sincedb_path => "/dev/null"
          start_position => "beginning"
          codec => multiline {
                  pattern => ".*"
                  what => "previous"
                  max_lines => 100000
                  max_bytes => "200 MiB"
          }
          type => "my-custom-type-1"
}

```

`sincedb_path` is set to /dev/null, it should not remember processed files, also tried setting `ignore_older` to 0, didn't help.

Also tried messing with queue settings in logstash.yml, changed it to persistent, didn't work ...

I'm using logstash version 7.5, logstash-input-file (4.1.11), running in linux machine.

When I restart logstash, then the unprocessed files get processed and cleaned up.  
I need it to work without restarting.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [December 14, 2019, 2:52pm UTC](https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854/2 "2019-12-14T14:52:54Z")

</div>

> [@konan\_pustolov](#):
>
> `sincedb_path` is set to /dev/null, it should not remember processed files, also tried setting `ignore_older` to 0, didn't help.

No, setting sincedb\_path to /dev/null prevents the file input persisting the sincedb across restarts. There is no way to prevent a file input remembering that it has processed a file.

Setting ignore\_older to zero tells a file input to ignore any files more than zero seconds old. It is completely different to the effect it has in filebeat.

---

<div class="post-metadata">

### Author: ![konan\_pustolov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/konan_pustolov/32/59375_2.png) [@konan\_pustolov](https://discuss.elastic.co/u/konan_pustolov)
#### Post date: [December 14, 2019, 6:13pm UTC](https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854/3 "2019-12-14T18:13:39Z")

</div>

Thanks for the response,

> [@Badger](#):
>
> There is no way to prevent a file input remembering that it has processed a file.

so what I need is not possible with the file input plugin.  
I'll try to dodge the problem somehow or try with some other input.

---

<div class="post-metadata">

### Author: ![VietCong](https://avatars.discourse-cdn.com/v4/letter/v/e47774/32.png) [@VietCong](https://discuss.elastic.co/u/VietCong)
#### Post date: [December 14, 2019, 7:10pm UTC](https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854/4 "2019-12-14T19:10:54Z")

</div>

You can also try pipeline-to-pipeline operation. Or do all the parsing in the logstash filter. I am sure you can parse message as many times as you like

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 11, 2020, 7:11pm UTC](https://discuss.elastic.co/t/logstash-file-input-not-reparsing-file/211854/5 "2020-01-11T19:11:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
