# How does sincedb work for a Logstash reading multiple files from a single directory?

**URL:** https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456
**Category:** Logstash
**Created:** [September 5, 2017, 3:15pm UTC](https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456 "2017-09-05T15:15:58Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [September 5, 2017, 3:15pm UTC](https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456/1 "2017-09-05T15:15:59Z")

</div>

Hi,

I am wondering how sincedb file works for multiple files in a single directory. When I start Logstash and once it process all the files sincedb file has only one line(probably just the information of a single file). But when I stop the Logstash, I see the sincedb file has the information about all files. I have a config file like,

```
file{
		path=>"C:/Documents/*.out"
		start_position=> "beginning"
		sincedb_path => "C:/logstash-5.3.1/logstash-5.3.1/data/plugins/inputs/file/sincedb.log"
		type=>"failure"
}

```

Please shed some light on this. Thank you.

---

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [September 6, 2017, 6:11pm UTC](https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456/2 "2017-09-06T18:11:49Z")

</div>

Can someone please help on this?

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [September 7, 2017, 5:41pm UTC](https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456/3 "2017-09-07T17:41:53Z")

</div>

Is the documentation not enough? [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#\_tracking\_of\_current\_position\_in\_watched\_files](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_tracking_of_current_position_in_watched_files)

Please tell us how we can improve the docs.

The file input holds a map of `inode` -\> `current-position` in memory. Every [`sincedb_write_interval`](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-sincedb_write_interval) the map is persisted to disk. On Logstash shut down, its also persisted. Some times the file input thread is blocked by the queue and it doesn't get round to persisting the map.

---

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [September 7, 2017, 7:07pm UTC](https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456/4 "2017-09-07T19:07:14Z")

</div>

Hi Guy,

For my above mentioned example, I am using a single sincedb file for multiple input files. So how would the sincedb file look? Would it have multiple entries(information about all the input files)?

Thanks

---

<div class="post-metadata">

### Author: ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [September 8, 2017, 4:50pm UTC](https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456/5 "2017-09-08T16:50:06Z")

</div>

Yes.

Something like this. 5 files were read, first number is the inode and the last number is the bytes read.

```auto
20670166 1 4 1549041625
16400297 1 4 107043
22063260 1 4 91
14993529 1 4 281500
13377766 1 4 281500

```

---

<div class="post-metadata">

### Author: ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)
#### Post date: [September 9, 2017, 11:36pm UTC](https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456/6 "2017-09-09T23:36:01Z")

</div>

Makes sense. Thank you 😊

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 7, 2017, 11:36pm UTC](https://discuss.elastic.co/t/how-does-sincedb-work-for-a-logstash-reading-multiple-files-from-a-single-directory/99456/7 "2017-10-07T23:36:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
