Here is my logstash.yml:
config.reload.automatic: true
dead_letter_queue.enable: true
http.host: 0.0.0.0
path.config: /home/logstash_user/logstash/conf.d/
path.logs: /var/log/logstash
pipeline.batch.size: 250
pipeline.workers: 8
queue.checkpoint.writes: 1
queue.type: persisted
And this is the directory of the persistent queue:
$ sudo ls -lah /home/logstash_user/logstash/data/queue/main/
total 12M
drwxr-xr-x 2 naas_logstash logstash_user 80 Feb 27 15:12 .
drwxr-xr-x 3 naas_logstash logstash_user 18 Feb 22 11:52 ..
-rw-r--r-- 1 naas_logstash logstash_user 34 Feb 27 15:12 checkpoint.head
-rw-r--r-- 1 naas_logstash logstash_user 0 Feb 22 11:52 .lock
-rw-r--r-- 1 naas_logstash logstash_user 64M Feb 27 15:12 page.862
-rw-r--r-- 1 naas_logstash logstash_user 4 Feb 22 11:52 .queue-version
It's using the default max page size of 64M and the default total queue size of 1GB. However, the queue never creates a tail page and only ever keeps the head page filled. I'm seeing slowness in processing input messages so shouldn't it be using the whole queue? Thank you!