I think I'm having an issue similar to this post: Persistent Queue Configuration question
What seems to happen is logstash performance will degrade the longer it's processing. I suspect that there is a specific message that is causing the entire page, or worker to hang. At the moment, I have a cronjob set to delete the queue and restart the process, but that doesn't seem to be an ideal solution.
I've run 'strings' on the pagefiles and can see lots of messages there. Is there a good way to see what the pages contain, and if they are ack'd by a worker? Or, are the pages supposed to be deleted after they are processed?