I have a pipeline FileBeat --> Logstash --> ElasticSearch. I want to know what FileBeat does in case it cannot access Logstash for a longer time period.
The application being monitored logs to a file, the current file is always named like example.log, it gets rotated every hour, the older versions are renamed to example.log.2020-02-11-11, example.log.2020-02-11-10 and so on. After 4 hours they also get compressed.
Now assuming that Logstash is not available for a long time period: my question/worry is that Filebeat would keep the file harvesters open, as long as it has not received an acknowledgment from Logstash. And that this could lead in memory/disk usage on the monitored machine.
I have found the option ignore_older in FileBeat's log input configuration. My question is whether it is enough to set this option? Assuming close_inactive has the default value of 5m.