# Is it redundant to use file input plugin with persistent queue?

**URL:** <https://discuss.elastic.co/t/is-it-redundant-to-use-file-input-plugin-with-persistent-queue/226297>\
**Category:** Logstash\
**Created:** [April 3, 2020, 12:33am UTC](https://discuss.elastic.co/t/is-it-redundant-to-use-file-input-plugin-with-persistent-queue/226297 "2020-04-03T00:33:56Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![danielyahn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielyahn/32/41575_2.png) [@danielyahn](https://discuss.elastic.co/u/danielyahn)\
**Post date:** [April 3, 2020, 12:33am UTC](https://discuss.elastic.co/t/is-it-redundant-to-use-file-input-plugin-with-persistent-queue/226297/1 "2020-04-03T00:33:56Z")

</div>

is it redundant to use file input plugin with persistent queue? Does persistent queue provide any additional delivery guarantee?

Let's say file input plugin uses

- read mode
- sinceDB

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 3, 2020, 1:44pm UTC](https://discuss.elastic.co/t/is-it-redundant-to-use-file-input-plugin-with-persistent-queue/226297/2 "2020-04-03T13:44:00Z")

</div>

I don't think it is redundant. A file input could read a small file, flush it to the pipeline, and then delete the file. That is not dependent upon the outputs writing the events to their destinations.

---

<div class="post-metadata">

**Author:** ![danielyahn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielyahn/32/41575_2.png) [@danielyahn](https://discuss.elastic.co/u/danielyahn)\
**Post date:** [April 3, 2020, 2:04pm UTC](https://discuss.elastic.co/t/is-it-redundant-to-use-file-input-plugin-with-persistent-queue/226297/3 "2020-04-03T14:04:24Z")

</div>

Good point about small file scenario.

But I still think that sincedb and persistent queue are playing similar roles: providing durability by writing to disk. I see the differences as well, sincedb keeps track of offset and PQ has actual data. But essentially, you're reading the data from disk (File Input) and writing back to disk (PQ), and reading it again (Filter/Output plugin).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2020, 2:04pm UTC](https://discuss.elastic.co/t/is-it-redundant-to-use-file-input-plugin-with-persistent-queue/226297/4 "2020-05-01T14:04:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.