# After restart, logstash doesn't read only new files

**URL:** https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012
**Category:** Logstash
**Created:** [December 15, 2021, 3:47pm UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012 "2021-12-15T15:47:19Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![khouloud](https://avatars.discourse-cdn.com/v4/letter/k/d6d6ee/32.png) [@khouloud](https://discuss.elastic.co/u/khouloud)
#### Post date: [December 15, 2021, 3:47pm UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012/1 "2021-12-15T15:47:19Z")

</div>

Hello, is it possible when we stop and restart logstash doesn't read all the files that match the path from the beginning and read only new files appended?

---

<div class="post-metadata">

### Author: ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)
#### Post date: [December 15, 2021, 3:50pm UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012/2 "2021-12-15T15:50:17Z")

</div>

Hi,

You can read this documentation about the sincedb path !

> **[File input plugin | Logstash Reference \[7.16\] | Elastic](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html)**

---

<div class="post-metadata">

### Author: ![khouloud](https://avatars.discourse-cdn.com/v4/letter/k/d6d6ee/32.png) [@khouloud](https://discuss.elastic.co/u/khouloud)
#### Post date: [December 15, 2021, 4:08pm UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012/3 "2021-12-15T16:08:47Z")

</div>

Can I use it for production?

---

<div class="post-metadata">

### Author: ![grumo35](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/grumo35/32/59451_2.png) [@grumo35](https://discuss.elastic.co/u/grumo35)
#### Post date: [December 15, 2021, 4:16pm UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012/4 "2021-12-15T16:16:17Z")

</div>

What do you mean ?

this plugin is a fully developped logstash plugin.

If you have a logstash input you're already using the input "plugin"

---

<div class="post-metadata">

### Author: ![khouloud](https://avatars.discourse-cdn.com/v4/letter/k/d6d6ee/32.png) [@khouloud](https://discuss.elastic.co/u/khouloud)
#### Post date: [December 16, 2021, 10:42am UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012/5 "2021-12-16T10:42:17Z")

</div>

Hello,  
I already use logstash and this is the input that I work on :

```auto
input {
  file {
    path => "/var/myapp/*/error.log"
    start_position => "beginning"
    type => "logs"
    sincedb_path => "/dev/null"
 }
}

```

- in the path : is a subdirectory, create everyday  
for example : today 1\_12\_2021  
/var/myapp/1\_12\_2021/error.log  
after running logstash this file is treated  
at 2\_12\_2021  
I had 2 subdirectories, each one contain an error file:  
1\_12\_2021 and 2\_12\_2021  
I restart logstash it will treat 2 files  
how can I configure logstash to make it read only from the new subdirectory?

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [December 16, 2021, 12:38pm UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012/6 "2021-12-16T12:38:20Z")

</div>

If you set `sincedb_path` to `/dev/null`, this will make logstash read ALL files when you restart because it is not tracking which file was already read.

Remove the `sincedb_path` from your configuration or change it to point to a file, something like `/var/lib/logstash/track-files.db`, so logstash can track what files it alread read.

---

<div class="post-metadata">

### Author: ![khouloud](https://avatars.discourse-cdn.com/v4/letter/k/d6d6ee/32.png) [@khouloud](https://discuss.elastic.co/u/khouloud)
#### Post date: [December 16, 2021, 1:12pm UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012/7 "2021-12-16T13:12:12Z")

</div>

I will try it  
Thank you

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 13, 2022, 1:12pm UTC](https://discuss.elastic.co/t/after-restart-logstash-doesnt-read-only-new-files/292012/8 "2022-01-13T13:12:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
