# Question: what happens when Logstash is unavailable for long?

**URL:** https://discuss.elastic.co/t/question-what-happens-when-logstash-is-unavailable-for-long/218759
**Category:** Beats
**Tags:** filebeat
**Created:** [February 11, 2020, 11:21am UTC](https://discuss.elastic.co/t/question-what-happens-when-logstash-is-unavailable-for-long/218759 "2020-02-11T11:21:48Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Christos\_Gitsis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christos_gitsis/32/56529_2.png) [@Christos\_Gitsis](https://discuss.elastic.co/u/Christos_Gitsis)
#### Post date: [February 11, 2020, 11:21am UTC](https://discuss.elastic.co/t/question-what-happens-when-logstash-is-unavailable-for-long/218759/1 "2020-02-11T11:21:48Z")

</div>

I have a pipeline FileBeat --\> Logstash --\> ElasticSearch. I want to know what FileBeat does in case it cannot access Logstash for a longer time period.

The application being monitored logs to a file, the current file is always named like example.log, it gets rotated every hour, the older versions are renamed to example.log.2020-02-11-11, example.log.2020-02-11-10 and so on. After 4 hours they also get compressed.

Now assuming that Logstash is not available for a long time period: my question/worry is that Filebeat would keep the file harvesters open, as long as it has not received an acknowledgment from Logstash. And that this could lead in memory/disk usage on the monitored machine.

I have found the option ignore\_older in FileBeat's log input configuration. My question is whether it is enough to set this option? Assuming close\_inactive has the default value of 5m.

---

<div class="post-metadata">

### Author: ![faec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faec/32/46988_2.png) [@faec](https://discuss.elastic.co/u/faec)
#### Post date: [February 13, 2020, 10:03pm UTC](https://discuss.elastic.co/t/question-what-happens-when-logstash-is-unavailable-for-long/218759/2 "2020-02-13T22:03:29Z")

</div>

Typically in this case Filebeat will continue read the incoming events until its queue is full (see queue settings [here](https://www.elastic.co/guide/en/beats/filebeat/master/configuring-internal-queue.html), the default size is 2048 events), then will pause harvesting until its backend is available, at which point it will resume. Logs that are deleted or rotated outside of Filebeat's target pattern in the meantime will be skipped.

Memory is probably only a concern if you're expecting very large events (which is possible but uncommon in plaintext logs), in which case you could lower the queue size. On the other hand, if the log entries are comparatively small you could _increase_ the queue size to reduce the amount of data lost when logstash is unavailable.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 12, 2020, 10:03pm UTC](https://discuss.elastic.co/t/question-what-happens-when-logstash-is-unavailable-for-long/218759/3 "2020-03-12T22:03:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
