I have recently enabled persistent queue configuration for logstash. I do see some files in location
/var/lib/logstash/queue/<pipeline_id>/, named as checkpoint.head and page.50. But, when i try to cat and read the files i am not able to understand in which format they are written.
So, trying to understand what this checkpoint.head and page files are?
How, do we know if the events received by input are stored in queue and are then processed to push in to elastic search.
And also which file system does logstash use for persistent queue in linux machine?
Any help in understanding the persistent queue and monitoring it's functionality is appreciated.
1 Like
Use the logstash monitoring api to see queue size.
All events are "persisted" to these files before logstash attempts to process them, so input is written to disk before processing (filter and output) is attempted.
I don't think looking at these files is a documented api 
Which filesystem is fromlogstash.yml path.data, usually /var/lib/logstash, at lest for RH Linux flavors.
I am not clear with the file system part you have mentioned.
The queue files are located in /var/lib/logstash/queue/ ...
But, which filesystem or disk does logstash use to write it events in to?
Logstash output is configured in various output plugins, usually logstash sends via tcp to an elasticsearch cluster.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.