DLQ Error: invalid checksum of record

Elastic Stack Logstash
1

I am trying to drop all messages from dlq.

input {
    dead_letter_queue {
        path => "/var/lib/logstash/dead_letter_queue" 
        commit_offsets => true 
        pipeline_id => "default-pipeline" 
    }
}
filter {
  drop {}
}
output {
    elasticsearch {
        hosts => [ "elk-es-data1", "elk-es-data2", "elk-es-data3", "elk-es-data4" ]
        user => admin
        password => admin
        index => '%{+YYYY.MM.dd}-dlq'
    }
}

It is working when I am using mutate filter and send it to index. But when I use drop {} filter and send small amount of data I receive error:

[2018-09-21T11:50:35,035][ERROR][logstash.pipeline        ] A plugin had an unrecoverable error. Will restart this plugin.
  Pipeline_id:dlq-pipeline
  Plugin: <LogStash::Inputs::DeadLetterQueue path=>"/var/lib/logstash/dead_letter_queue", commit_offsets=>true, pipeline_id=>"default-pipeline", id=>"8d6e92c5397fe4e6882f3e9db868ec48f671b64f5a1dacee706adfc73a2e5523", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_0f81d711-8d8d-401a-8ed1-cc51c79ce575", enable_metric=>true, charset=>"UTF-8">>
  Error: invalid checksum of record
  Exception: Java::JavaLang::RuntimeException
  Stack: org.logstash.common.io.RecordIOReader.getRecord(org/logstash/common/io/RecordIOReader.java:217)
org.logstash.common.io.RecordIOReader.readEvent(org/logstash/common/io/RecordIOReader.java:233)
org.logstash.common.io.DeadLetterQueueReader.pollEntryBytes(org/logstash/common/io/DeadLetterQueueReader.java:120)
org.logstash.common.io.DeadLetterQueueReader.pollEntry(org/logstash/common/io/DeadLetterQueueReader.java:97)
org.logstash.input.DeadLetterQueueInputPlugin.run(org/logstash/input/DeadLetterQueueInputPlugin.java:82)
java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:423)
org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:290)
RUBY.run(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-dead_letter_queue-1.1.4/lib/logstash/inputs/dead_letter_queue.rb:60)
RUBY.inputworker(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:514)
RUBY.block in start_input(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:507)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:289)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:246)
java.lang.Thread.run(java/lang/Thread.java:748)

Is it problem with my configuration or is it a bug? How should I resolve this issue?

logstash.version=> 6.2.4

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.