How to deal with the window when logstash is down and logs may be lost


(Shuo) #1

The situation is like this.
I run a logstash docker to monitor the log file example.log and logstash consumes the logs to Elasticsearch.
There is a rotation for log files. When the file is large enough it is archived and the application only keeps 10 archived files and deletes the rest.
Now there might be a window when logstash is down and the unconsumed logs are archived even deleted. So we might lose those logs.
Since logstash can not remove files after consuming them, I can only use log rotation.
Is there some good way to solve this problem?


(Shuo) #2

Is there someone who has better ideas?


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.