Hello,
I currently have logstash running within a docker container and when I have it configured for persistent queue it works as it should. The only issue is when I mount a volume to where the persistent queue directory resides in the vm the logstash pipeline will not start. It seems as if the paging mechinism will not work if you mount a volume to where the queues exist and it sucks because if the docker VM has a critical failure then events that were saved to the persistent queues are lost! Thus the reason why having it mount to a volume on the host is essential. looks like the offended class is: JrubyAckedQueueExtLibrary.java. Anyone else ran into this type of issue? And FYI- Mounting a volume to the dead_letter_queue works perfectly fine, the .sincedb files and queue files get copied to the host with no exceptions.
logstash.yml
queue.type: persisted
path.queue: /opt/logstash/persistance/data
queue.max_events: 0
queue.max_bytes: 10240mb
queue.checkpoint.acks: 1024
queue.checkpoint.writes: 1024
queue.checkpoint.interval: 1000
Logstash Exception
[2017-09-19T20:07:33,452][ERROR][logstash.pipeline ] Logstash failed to create queue {"exception"=>"Invalid argument", "backtrace"=>["org/logstash/ackedqueue/ext/JrubyAckedQueueExtLibrary.java:133:in open'", "/opt/logstash/logstash-core/lib/logstash/util/wrapped_acked_queue.rb:41:inwith_queue'", "/opt/logstash/logstash-core/lib/logstash/util/wrapped_acked_queue.rb:30:in create_file_based'", "/opt/logstash/logstash-core/lib/logstash/queue_factory.rb:29:increate'", "/opt/logstash/logstash-core/lib/logstash/pipeline.rb:168:in initialize'", "/opt/logstash/logstash-core/lib/logstash/agent.rb:286:increate_pipeline'", "/opt/logstash/logstash-core/lib/logstash/agent.rb:95:in register_pipeline'", "/opt/logstash/logstash-core/lib/logstash/runner.rb:309:inexecute'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/opt/logstash/logstash-core/lib/logstash/runner.rb:204:inrun'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/opt/logstash/lib/bootstrap/environment.rb:71:in(root)'"]}
[2017-09-19T20:07:33,458][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Invalid argument"}
[
