# Read mode input plugin and sincedb

**URL:** https://discuss.elastic.co/t/read-mode-input-plugin-and-sincedb/220843
**Category:** Logstash
**Created:** [February 25, 2020, 11:54am UTC](https://discuss.elastic.co/t/read-mode-input-plugin-and-sincedb/220843 "2020-02-25T11:54:10Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![akwoprosper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akwoprosper/32/45810_2.png) [@akwoprosper](https://discuss.elastic.co/u/akwoprosper)
#### Post date: [February 25, 2020, 11:54am UTC](https://discuss.elastic.co/t/read-mode-input-plugin-and-sincedb/220843/1 "2020-02-25T11:54:10Z")

</div>

Hello everyone,

I am presently using logstash to ingest billions of records into elasticsearch.

I am using the file input plugin to read the files with the input mode set to read since all the data is already present.

The problem is that, each file is very large(close to 30GB) containing 200million records on average . This means that logstash will need much time to process each of these files.

I just noticed that logstash logs data into the since\_db file only after completely reading a file, which is problematic given that a file may be read multiple times if logstash is restarted.

**Questions**

- Is my observation correct
- How do I configure logstash to write to the since\_db file more often so that it continues from where it left off in case logstash restarts

Thanks in advance

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 24, 2020, 11:54am UTC](https://discuss.elastic.co/t/read-mode-input-plugin-and-sincedb/220843/2 "2020-03-24T11:54:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
