# Persistent Queues delay output
**URL:** https://discuss.elastic.co/t/persistent-queues-delay-output/123255
**Category:** Logstash
**Created:** [March 9, 2018, 11:27am UTC](https://discuss.elastic.co/t/persistent-queues-delay-output/123255 "2018-03-09T11:27:23Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Hai](https://discuss.elastic.co/u/Hai)
#### Post date: [March 9, 2018, 11:27am UTC](https://discuss.elastic.co/t/persistent-queues-delay-output/123255/1 "2018-03-09T11:27:23Z")
when logstash enable `queue.type: persisted`, other parameters is default, i found logstash output very slow, but queue has buffered hundreds of megabytes of data. from visualize i got follow pic, normally data jitter frequency is not large.
---
### Author:  [@PandKing](https://discuss.elastic.co/u/PandKing)
#### Post date: [March 9, 2018, 2:09pm UTC](https://discuss.elastic.co/t/persistent-queues-delay-output/123255/2 "2018-03-09T14:09:09Z")
I was looking at the same info. It doesn't make sense to me why I am Queuing so much when the output isn't that busy. To the point it just keeps growing. Anybody have a good best practice ?
[2018-03-08T09:40:44,689][DEBUG][logstash.runner] -------- Logstash Settings (\* means modified) ---------
[2018-03-08T09:40:44,689][DEBUG][logstash.runner] node.name: "[ent-mocsmlh01.us.ad.gannett.com](http://ent-mocsmlh01.us.ad.gannett.com)."
[2018-03-08T09:40:44,689][DEBUG][logstash.runner] \*path.config: "/etc/logstash/conf.d"
[2018-03-08T09:40:44,689][DEBUG][logstash.runner] \*path.data: "/var/lib/logstash" (default: "/usr/share/logstash/data")
[2018-03-08T09:40:44,689][DEBUG][logstash.runner] modules.cli: []
[2018-03-08T09:40:44,689][DEBUG][logstash.runner] modules: []
[2018-03-08T09:40:44,689][DEBUG][logstash.runner] modules\_setup: false
[2018-03-08T09:40:44,690][DEBUG][logstash.runner] config.test\_and\_exit: false
[2018-03-08T09:40:44,690][DEBUG][logstash.runner] config.reload.automatic: false
[2018-03-08T09:40:44,690][DEBUG][logstash.runner] config.reload.interval: 3000000000
[2018-03-08T09:40:44,693][DEBUG][logstash.runner] config.support\_escapes: false
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] metric.collect: true
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] pipeline.id: "main"
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] pipeline.system: false
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] \*pipeline.workers: 6 (default: 2)
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] \*pipeline.output.workers: 2 (default: 1)
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] pipeline.batch.size: 125
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] pipeline.batch.delay: 50
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] pipeline.unsafe\_shutdown: false
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] pipeline.java\_execution: false
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] pipeline.reloadable: true
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] path.plugins: []
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] config.debug: false
[2018-03-08T09:40:44,694][DEBUG][logstash.runner] \*log.level: "debug" (default: "info")
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] version: false
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] help: false
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] log.format: "plain"
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] http.host: "127.0.0.1"
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] http.port: 9600..9700
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] http.environment: "production"
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] \*queue.type: "persisted" (default: "memory")
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] queue.drain: false
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] queue.page\_capacity: 67108864
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] queue.max\_bytes: 1073741824
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] \*queue.max\_events: 2048 (default: 0)
[2018-03-08T09:40:44,695][DEBUG][logstash.runner] queue.checkpoint.acks: 1024
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] queue.checkpoint.writes: 1024
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] queue.checkpoint.interval: 1000
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] dead\_letter\_queue.enable: false
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] dead\_letter\_queue.max\_bytes: 1073741824
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] slowlog.threshold.warn: -1
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] slowlog.threshold.info: -1
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] slowlog.threshold.debug: -1
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] slowlog.threshold.trace: -1
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] keystore.classname: "org.logstash.secret.store.backend.JavaKeyStore"
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] \*keystore.file: "/etc/logstash/logstash.keystore" (default: "/usr/share/logstash/config/logstash.keystore")
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] \*path.queue: "/var/lib/logstash/queue" (default: "/usr/share/logstash/data/queue")
[2018-03-08T09:40:44,696][DEBUG][logstash.runner] \*path.dead\_letter\_queue: "/var/lib/logstash/dead\_letter\_queue" (default: "/usr/share/logstash/data/dead\_letter\_queue")
[2018-03-08T09:40:44,697][DEBUG][logstash.runner] \*path.settings: "/etc/logstash" (default: "/usr/share/logstash/config")
[2018-03-08T09:40:44,697][DEBUG][logstash.runner] \*path.logs: "/var/log/logstash" (default: "/usr/share/logstash/logs")
[2018-03-08T09:40:44,697][DEBUG][logstash.runner] --------------- Logstash Settings -------------------
---
### Author:  [@PandKing](https://discuss.elastic.co/u/PandKing)
#### Post date: [March 12, 2018, 9:36pm UTC](https://discuss.elastic.co/t/persistent-queues-delay-output/123255/3 "2018-03-12T21:36:38Z")
I seem to have fixed some of my problem with tuning my http output re-try.
Switched back to memory before I added these but may have the same effect on persistent Queues. I was using the SumoLogic http output plugin and 3Gb of heap which ran out in \<9 hrs.
Now my heap isn't growing above 20% used.
automatic\_retries =\> 10
validate\_after\_inactivity =\> 3
request\_timeout =\> 10
---
### Author:  [@Hai](https://discuss.elastic.co/u/Hai)
#### Post date: [March 13, 2018, 3:47am UTC](https://discuss.elastic.co/t/persistent-queues-delay-output/123255/4 "2018-03-13T03:47:12Z")
em..., i use elasticesearch output plugin, there seems to be no suitable option for this case.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 10, 2018, 3:47am UTC](https://discuss.elastic.co/t/persistent-queues-delay-output/123255/5 "2018-04-10T03:47:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.