I tried dead letter queue plugin but it's not giving consistent output. and also i am facing the below issues.
Error:
[2018-10-16T06:56:42,248][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Plugin: <LogStash::Inputs::DeadLetterQueue path=>"opt/logstash/logstash-5.5.0/data/dead_letter_queue", commit_offsets=>true, pipeline_id=>"main", id=>"b49487305765c7531f10d6f3df975a1e2893844f-3", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_e6699c41-029b-4009-9775-1be9592d29e4", enable_metric=>true, charset=>"UTF-8">>
[2018-11-09T05:09:31,671][DEBUG][logstash.agent ] Reading config file {:config_file=>"/etc/logstash/conf/logstash.conf"}
[2018-11-09T05:09:31,671][DEBUG][logstash.agent ] no configuration change for pipeline {:pipeline=>"main"}
[2018-11-09T05:09:31,926][DEBUG][logstash.instrument.periodicpoller.cgroup] Error, cannot retrieve cgroups information {:exception=>"Errno::ENOENT", :message=>"No such file or directory - /sys/fs/cgroup/cpuacct/ecs/9dc36c1f-a3a7-4265-b9b1-80dee87be626/e29b6c7b3eb67a8ea956384687baf66ebf3974f0476b5c74671684c5815362ac/cpuacct.usage"}
[2018-11-09T05:09:32,512][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Plugin: <LogStash::Inputs::DeadLetterQueue path=>"opt/logstash/logstash-5.5.0/data/dead_letter_queue", commit_offsets=>true, pipeline_id=>"main", id=>"7d2f7b47afb2aafe313f6f4a2411d56d61ba9e13-3", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_f2ba8af4-e928-4329-8754-8804a048136e", enable_metric=>true, charset=>"UTF-8">>
Error:
Exception: Java::JavaLang::NullPointerException
Stack: java.nio.channels.FileChannel.open(java/nio/channels/FileChannel.java:286)
java.nio.channels.FileChannel.open(java/nio/channels/FileChannel.java:335)
org.logstash.common.io.RecordIOReader.(org/logstash/common/io/RecordIOReader.java:52)
org.logstash.common.io.DeadLetterQueueReader.pollEntryBytes(org/logstash/common/io/DeadLetterQueueReader.java:124)
org.logstash.common.io.DeadLetterQueueReader.pollEntry(org/logstash/common/io/DeadLetterQueueReader.java:95)
org.logstash.input.DeadLetterQueueInputPlugin.run(org/logstash/input/DeadLetterQueueInputPlugin.java:82)
java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
LogStash::Inputs::DeadLetterQueue.run(/opt/logstash/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-input-dead_letter_queue-1.1.4/lib/logstash/inputs/dead_letter_queue.rb:60)
LogStash::Inputs::DeadLetterQueue.run(/opt/logstash/logstash-5.5.0/vendor/bundle/jruby/1.9/gems/logstash-input-dead_letter_queue-1.1.4/lib/logstash/inputs/dead_letter_queue.rb:60)
RUBY.inputworker(/opt/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:456)
RUBY.start_input(/opt/logstash/logstash-5.5.0/logstash-core/lib/logstash/pipeline.rb:449)