# Logstash Error - Deadletter queue/plugin - troubleshooting

**URL:** <https://discuss.elastic.co/t/logstash-error-deadletter-queue-plugin-troubleshooting/160250>\
**Category:** Logstash\
**Created:** [December 10, 2018, 10:07pm UTC](https://discuss.elastic.co/t/logstash-error-deadletter-queue-plugin-troubleshooting/160250 "2018-12-10T22:07:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Karthik\_Ramachandran](https://avatars.discourse-cdn.com/v4/letter/k/f19dbf/32.png) [@Karthik\_Ramachandran](https://discuss.elastic.co/u/Karthik_Ramachandran)\
**Post date:** [December 10, 2018, 10:07pm UTC](https://discuss.elastic.co/t/logstash-error-deadletter-queue-plugin-troubleshooting/160250/1 "2018-12-10T22:07:36Z")

</div>

Dear All,

Trying to trouble shoot the logstash issues that I'm facing, and any hints would be helpful

Background

Note the issue is in both prod and dev, and the numbers below are from dev  
Log ES Version : 6.2.1  
Monitoring ES Version : 6.3.0  
Logstash version: [docker.elastic.co:443/logstash/logstash](http://docker.elastic.co:443/logstash/logstash) 6.2.4

Both logging elastic, and logstash send monitoring data to Monitoring ES.

Problems

1. During logstash setup, we get the below error

The logstash is in multi-pipeline mode, and we have the deadletter queues configured using a separate pipeline "deadletter.conf" with appropriate inputs/outputs representing all other pipelines. The deadletter pipeline, which runs as "main" sends the deadletter data to ES Log against a separate indice with no strict mapping. The error indicates that one of the pipeline is missing. Is looks like it is due to the fact that deadletter pipeline is also started along with other pipeline initially. This comes only during logstash provisioning and start after deploying all pipelines. Not sure if the observation is correct.

1. Restart Logstash (docker instance), and below error is observed during restart. Not sure why this error is coming for each pipeline that is getting recoginized. Finally all pipelines including "main" that represent deadletter is running.  
[ERROR] 2018-12-10 21:26:19.002 [[main]\<dead\_letter\_queue] pipeline - A plugin had an unrecoverable error. Will restart this plugin.  
Pipeline\_id:main  
Plugin: \<LogStash::Inputs::DeadLetterQueue path=\>"/var/log/logstash/deadletter", commit\_offsets=\>true, pipeline\_id=\>" **FOR EACH PIPELINE**", id=\>"8642a2efb7b01a2d283c862ad0d50b7eaee3e354425bd3026467d66af5546790", enable\_metric=\>true, codec=\>\<LogStash::Codecs::Plain id=\>"plain\_4866de30-3254-43fb-9a9a-0ea75ec572a1", enable\_metric=\>true, charset=\>"UTF-8"\>\>  
Error: Invalid file. check version  
Exception: Java::JavaLang::RuntimeException  
Stack: org.logstash.common.io.RecordIOReader.(org/logstash/common/io/RecordIOReader.java:59)  
org.logstash.common.io.DeadLetterQueueReader.pollEntryBytes(org/logstash/common/io/DeadLetterQueueReader.java:126)  
org.logstash.common.io.DeadLetterQueueReader.pollEntry(org/logstash/common/io/DeadLetterQueueReader.java:97)  
org.logstash.input.DeadLetterQueueInputPlugin.run(org/logstash/input/DeadLetterQueueInputPlugin.java:82)  
java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498)  
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:423)  
org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:290)  
RUBY.run(/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-dead\_letter\_queue-1.1.4/lib/logstash/inputs/dead\_letter\_queue.rb:60)  
usr.share.logstash.logstash\_minus\_core.lib.logstash.pipeline.invokeOther8:run(usr/share/logstash/logstash\_minus\_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:514)  
usr.share.logstash.logstash\_minus\_core.lib.logstash.pipeline.inputworker(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:514)  
usr.share.logstash.logstash\_minus\_core.lib.logstash.pipeline.invokeOther0:inputworker(usr/share/logstash/logstash\_minus\_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:507)  
usr.share.logstash.logstash\_minus\_core.lib.logstash.pipeline.block in start\_input(/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:507)  
org.jruby.RubyProc.call(org/jruby/RubyProc.java:289)  
org.jruby.RubyProc.call(org/jruby/RubyProc.java:246)  
java.lang.Thread.run(java/lang/Thread.java:748)

Any help in identifying the cause and recover from these errors would be helpful

---

<div class="post-metadata">

**Author:** ![Karthik\_Ramachandran](https://avatars.discourse-cdn.com/v4/letter/k/f19dbf/32.png) [@Karthik\_Ramachandran](https://discuss.elastic.co/u/Karthik_Ramachandran)\
**Post date:** [January 2, 2019, 9:21pm UTC](https://discuss.elastic.co/t/logstash-error-deadletter-queue-plugin-troubleshooting/160250/2 "2019-01-02T21:21:45Z")

</div>

Rechecking as I'm still facing the issue ☹

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 30, 2019, 9:21pm UTC](https://discuss.elastic.co/t/logstash-error-deadletter-queue-plugin-troubleshooting/160250/3 "2019-01-30T21:21:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.