Persistent queue only filling on page

Here is my logstash.yml:

config.reload.automatic: true
dead_letter_queue.enable: true
http.host: 0.0.0.0
path.config: /home/logstash_user/logstash/conf.d/
path.logs: /var/log/logstash
pipeline.batch.size: 250
pipeline.workers: 8
queue.checkpoint.writes: 1
queue.type: persisted

And this is the directory of the persistent queue:

$ sudo ls -lah /home/logstash_user/logstash/data/queue/main/
total 12M
drwxr-xr-x 2 naas_logstash logstash_user 80 Feb 27 15:12 .
drwxr-xr-x 3 naas_logstash logstash_user 18 Feb 22 11:52 ..
-rw-r--r-- 1 naas_logstash logstash_user 34 Feb 27 15:12 checkpoint.head
-rw-r--r-- 1 naas_logstash logstash_user 0 Feb 22 11:52 .lock
-rw-r--r-- 1 naas_logstash logstash_user 64M Feb 27 15:12 page.862
-rw-r--r-- 1 naas_logstash logstash_user 4 Feb 22 11:52 .queue-version

It's using the default max page size of 64M and the default total queue size of 1GB. However, the queue never creates a tail page and only ever keeps the head page filled. I'm seeing slowness in processing input messages so shouldn't it be using the whole queue? Thank you!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.