# Logstash: error in processing events from DLQ

**URL:** <https://discuss.elastic.co/t/logstash-error-in-processing-events-from-dlq/125406>\
**Category:** Logstash\
**Created:** [March 23, 2018, 7:25pm UTC](https://discuss.elastic.co/t/logstash-error-in-processing-events-from-dlq/125406 "2018-03-23T19:25:19Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pandeesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pandeesh/32/29194_2.png) [@pandeesh](https://discuss.elastic.co/u/pandeesh)\
**Post date:** [March 23, 2018, 7:25pm UTC](https://discuss.elastic.co/t/logstash-error-in-processing-events-from-dlq/125406/1 "2018-03-23T19:25:19Z")

</div>

All,

I have configured my primary pipeline to process events from beats plugin and write it to Dead Letter Queue whenever there're any issues.  
Then I have a secondary pipeline to read from the DLQ and send it to Elastic Search in a separate index.  
secondary pipeline:

```
  input {

 dead_letter_queue {
          path => "/var/lib/logstash/dead_letter_queue"
         commit_offsets => true
        codec => "json"
  }
  }

 output {

elasticsearch {
        document_type => "dlq"
        hosts => ["es host"]
        index => "dlq-%{+YYYY.MM.dd}"
  }

 }

```

When i enabled this , it's throwing:

> Pipeline aborted due to error {:exception=\>java.nio.file.NoSuchFileException: /var/lib/logstash/dead\_letter\_queue/main/60.log, :backtrace=\>["sun.nio.fs.UnixException.translateToIOException(sun/nio/fs/UnixException.java:86)", "sun.nio.fs.UnixException.rethrowAsIOException(sun/nio/fs/UnixException.java:102)", "sun.nio.fs.UnixException.rethrowAsIOException(sun/nio/fs/UnixException.java:107)"

I am wondering whether there's a caching behavior on DLQ plugin which looks for the file which was being processed last time and resume from there? currently I don't have any file called 60.log in the referenced path in the error.  
I tried setting both commit\_offsets =\> true / false, still the same error persists.  
Is there a way I can completely reset the DLQ plugin to start from the scratch?

Logstash version: logstash 5.5.0

Thanks for helping

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [March 24, 2018, 8:28am UTC](https://discuss.elastic.co/t/logstash-error-in-processing-events-from-dlq/125406/2 "2018-03-24T08:28:07Z")

</div>

I found a related fix that is included in 5.6.3 and beyond:

> <https://github.com/elastic/logstash/pull/8397>

---

<div class="post-metadata">

**Author:** ![pandeesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pandeesh/32/29194_2.png) [@pandeesh](https://discuss.elastic.co/u/pandeesh)\
**Post date:** [March 25, 2018, 12:14am UTC](https://discuss.elastic.co/t/logstash-error-in-processing-events-from-dlq/125406/3 "2018-03-25T00:14:46Z")

</div>

Thanks Yaauie. I will take a look.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 22, 2018, 12:14am UTC](https://discuss.elastic.co/t/logstash-error-in-processing-events-from-dlq/125406/4 "2018-04-22T00:14:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.