# Store incoming data before dealing with it?

**URL:** <https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349>\
**Category:** Logstash\
**Created:** [May 11, 2017, 8:26am UTC](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349 "2017-05-11T08:26:34Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![BrunoC](https://avatars.discourse-cdn.com/v4/letter/b/2acd7d/32.png) [@BrunoC](https://discuss.elastic.co/u/BrunoC)\
**Post date:** [May 11, 2017, 8:26am UTC](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349/1 "2017-05-11T08:26:35Z")

</div>

Hey there,

I'm getting data to Logstash from tcp and udp but I want to store these data into a file before Logstash process it. Is this possible?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 11, 2017, 10:33am UTC](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349/2 "2017-05-11T10:33:13Z")

</div>

The new persistent queue functionality allows data to be stored on disk while it is processed. This allows Logstash to buffer if there are issues with output destinations. Does this match what you are looking for or is there a different reason behind your question?

---

<div class="post-metadata">

**Author:** ![BrunoC](https://avatars.discourse-cdn.com/v4/letter/b/2acd7d/32.png) [@BrunoC](https://discuss.elastic.co/u/BrunoC)\
**Post date:** [May 11, 2017, 10:44am UTC](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349/3 "2017-05-11T10:44:06Z")

</div>

It kind of answer the question. I'm getting logs from pfSense through tcp/udp to Logstash but I want to store these logs because I need to do some other operation on them that is totally irrelevant to Logstash.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 11, 2017, 11:31am UTC](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349/4 "2017-05-11T11:31:18Z")

</div>

Logstash has a file output plugin that does allow you to write data to file.

---

<div class="post-metadata">

**Author:** ![BrunoC](https://avatars.discourse-cdn.com/v4/letter/b/2acd7d/32.png) [@BrunoC](https://discuss.elastic.co/u/BrunoC)\
**Post date:** [May 11, 2017, 11:35am UTC](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349/5 "2017-05-11T11:35:25Z")

</div>

I tried it and it writes out the data that has been processed by Logstash (there is already all Logstash's filters).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 8, 2017, 11:35am UTC](https://discuss.elastic.co/t/store-incoming-data-before-dealing-with-it/85349/6 "2017-06-08T11:35:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.