The persistent queue can be configured to hold a specific number of events, and it can be limited in size (see persistent queue doc) , the parameters are called
queue.max_bytes. Due to regulations, we have to ensure that we do not store events older than x minutes. It would be great to have an age-based cleanup, and a parameter
queue.max_age. It would be good enough to use the timestamp of the event when it entered the queue.
One workaround idea that a collegue came up with is to delete the queue pages on disk after x minutes. But we didn't test how logstash would take that.
The information of how much time the event spent in logstash seems to exist in
duration_in_millis , but I'm not sure when exactly that starts and stops.