# DeadLetterQueue Configration

**URL:** <https://discuss.elastic.co/t/deadletterqueue-configration/324869>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [February 7, 2023, 7:55am UTC](https://discuss.elastic.co/t/deadletterqueue-configration/324869 "2023-02-07T07:55:40Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Venkata\_Sai\_K](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/venkata_sai_k/32/116851_2.png) [@Venkata\_Sai\_K](https://discuss.elastic.co/u/Venkata_Sai_K)\
**Post date:** [February 7, 2023, 7:55am UTC](https://discuss.elastic.co/t/deadletterqueue-configration/324869/1 "2023-02-07T07:55:40Z")

</div>

I have done Deadletter queue configration in one file and given to that as volume for the docker service , everything we have dockerized it and here are my files  
**persistent-queue and dlq config**  
pipeline.batch.size: 125

pipeline.batch.delay: 50

queue.type: persisted

path.queue: path.data/queue

queue.checkpoint.acks: 0

queue.checkpoint.writes: 0

queue.checkpoint.interval: 0

dead\_letter\_queue.enable: true

path.data: /usr/share/logstash/data

path.dead\_letter\_queue: "/usr/share/logstash/data/dead\_letter\_queue"

path.config: "/usr/share/logstash/pipeline/dead\_letter\_queue.conf"

dead\_letter\_queue.flush\_interval: 5000

dead\_letter\_queue.retain.age: 2d

**Dlq.conf**  
input {  
dead\_letter\_queue {  
path =\> "/usr/share/logstash/data/dead\_letter\_queue"  
commit\_offsets =\> true  
pipeline\_id =\> "main"  
clean\_consumed =\> true  
}  
}

output {  
stdout {  
codec =\> rubydebug {  
metadata =\> true # This makes the logs to appear in console  
}  
}  
opensearch {  
hosts =\> ["${LOGSTASH\_OPENSEARCH\_HOST}"]  
ssl =\> true  
ssl\_certificate\_verification =\> false  
user =\> "${LOGSTASH\_OPENSEARCH\_USER}"  
password =\> "${LOGSTASH\_OPENSEARCH\_PASSWORD}"  
index =\> "arraysofcars"  
document\_id =\> "%{id}"  
ecs\_compatibility =\> disabled

```
}

```

}  
**yml** file  
version: '3'  
services:  
db:  
container\_name: mysql\_container  
image: mysql:5.7  
restart: always  
environment:  
MYSQL\_ROOT\_PASSWORD: "root"  
MYSQL\_USER: "yash"  
MYSQL\_PASSWORD: "password"  
MYSQL\_DATABASE: "testDb"  
ports:  
- "3306:3306"  
volumes:  
- mysqldata:/var/lib/mysql  
networks:  
- share-network  
elasticsearch:  
image: amazon/opendistro-for-elasticsearch:1.13.2  
container\_name: elasticsearch  
environment:  
- bootstrap.memory\_lock=true  
- cluster.name=odfe-cluster  
- discovery.type=single-node  
- XPACK\_MONITORING\_ENABLED=true  
- "ES\_JAVA\_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM  
ulimits:  
memlock:  
soft: -1  
hard: -1  
nofile:  
soft: 65536 # maximum number of open files for the Elasticsearch user, set to at least 65536 on modern systems  
hard: 65536  
volumes:  
- odfe-data5:/usr/share/elasticsearch/data  
ports:  
- 9200:9200  
- 9600:9600 # required for Performance Analyzer  
networks:  
- share-network

kibana:  
image: opensearchproject/opensearch-dashboards:latest  
container\_name: odfe-kibana5  
ports:  
- 5601:5601  
environment:  
OPENSEARCH\_HOSTS: '["[https://elasticsearch:9200](https://elasticsearch:9200)"]'  
DISABLE\_SECURITY\_DASHBOARDS\_PLUGIN : "true"  
volumes:  
- ./kibana.yml:/usr/share/kibana/config/kibana.yml  
networks:  
- share-network

logstash:  
# image: napses/mvp-logstash:latest  
build:  
context: .  
dockerfile: ./Dockerfile  
container\_name: mvp\_logstash  
environment:  
- LOGSTASH\_JDBC\_URL=jdbc:mysql://db:3306/testDb?useSSL=false  
- LOGSTASH\_JDBC\_DRIVER=com.mysql.cj.jdbc.Driver  
- LOGSTASH\_JDBC\_USERNAME=yash  
- LOGSTASH\_JDBC\_PASSWORD=password  
- LOGSTASH\_OPENSEARCH\_HOST=[https://elasticsearch:9200](https://elasticsearch:9200)  
- LOGSTASH\_OPENSEARCH\_USER=admin  
- LOGSTASH\_OPENSEARCH\_PASSWORD=admin  
- XPACK\_SECURITY\_ENABLED=false  
- XPACK\_REPORTING\_ENABLED=false  
- XPACK\_MONITORING\_ENABLED=false  
volumes:  
- ./pipelines/mysql\_pipelines.conf:/usr/share/logstash/pipeline/logstash.conf  
- ./pipelines/dead\_letter\_queue.conf:/usr/share/logstash/pipeline/dead\_letter\_queue.conf  
- ./pq-dlq.yml:/usr/share/pq-dlq/config/pq-dlq.yml  
ports:  
- 9300:9300  
- 5044:5044  
networks:  
- share-network  
depends\_on:  
- elasticsearch  
- kibana  
- db

volumes:  
odfe-data5:  
mysqldata: {}

networks:  
share-network:  
driver: bridge  
and i am getting this errror  
Pipeline\_id:main  
mvp\_logstash | Plugin: \<LogStash::Inputs::DeadLetterQueue pipeline\_id=\>"main", clean\_consumed=\>true, path=\>"/usr/share/logstash/data/dead\_letter\_queue", id=\>"103d70998afc3a81ca902851533b56b646779ccdf856eed2189371fb6d44003a", commit\_offsets=\>true, enable\_metric=\>true, codec=\>\<LogStash::Codecs::Plain id=\>"plain\_71995489-b604-4ba1-b411-9adf60d53a11", enable\_metric=\>true, charset=\>"UTF-8"\>\>  
mvp\_logstash | Error: DLQ sub-path /usr/share/logstash/data/dead\_letter\_queue/main does not exist  
mvp\_logstash | Exception: Java::JavaNioFile::NoSuchFileException  
mvp\_logstash | Stack: org.logstash.input.DeadLetterQueueInputPlugin.lazyInitQueueReader(org/logstash/input/DeadLetterQueueInputPlugin.java:81)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 7, 2023, 7:55am UTC](https://discuss.elastic.co/t/deadletterqueue-configration/324869/2 "2023-02-07T07:55:40Z")

</div>

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 7, 2023, 7:56am UTC](https://discuss.elastic.co/t/deadletterqueue-configration/324869/3 "2023-03-07T07:56:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.