# Dead\_letter\_queue

**URL:** <https://discuss.elastic.co/t/dead-letter-queue/174063>\
**Category:** Logstash\
**Created:** [March 27, 2019, 8:29am UTC](https://discuss.elastic.co/t/dead-letter-queue/174063 "2019-03-27T08:29:46Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![xavi23](https://avatars.discourse-cdn.com/v4/letter/x/db5fbb/32.png) [@xavi23](https://discuss.elastic.co/u/xavi23)\
**Post date:** [March 27, 2019, 8:29am UTC](https://discuss.elastic.co/t/dead-letter-queue/174063/1 "2019-03-27T08:29:46Z")

</div>

Hello,

Actually I'm using a cluster of Amazon ElasticSearch and I'm trying to load some documents from S3 (input) using logstash.

I've one pipeline (file.conf) with 25 inputs , one for each different folder in S3.

The ouput is AmazonElasticSearch.

When logstash run I can see in my index that logstash yes index the documents but sometimes I Can see that not all documents.

I try to enable dead\_letter\_queue to know what queues anda data I'm lossing , but never see anything.

I change this parameters in my logstash.yml:

dead\_letter\_queue.enable: true

queue.type: persisted

I'm not sure if I need change another configuration or something.. ?

Thanks for advanced.

King regards.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 27, 2019, 8:33am UTC](https://discuss.elastic.co/t/dead-letter-queue/174063/2 "2019-03-27T08:33:57Z")

</div>

I believe not all output plugins support the dead letter queue, and it is possible that the `amazon_es` plugin (if that is what you are using) is one of them.

---

<div class="post-metadata">

**Author:** ![xavi23](https://avatars.discourse-cdn.com/v4/letter/x/db5fbb/32.png) [@xavi23](https://discuss.elastic.co/u/xavi23)\
**Post date:** [March 27, 2019, 8:59am UTC](https://discuss.elastic.co/t/dead-letter-queue/174063/3 "2019-03-27T08:59:43Z")

</div>

Yes I'm using amazon\_es. How I can check if this ouput accept dead\_letter\_queue ?

Thanks

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 27, 2019, 10:03am UTC](https://discuss.elastic.co/t/dead-letter-queue/174063/4 "2019-03-27T10:03:47Z")

</div>

I would recommend you check with Amazon.

---

<div class="post-metadata">

**Author:** ![xavi23](https://avatars.discourse-cdn.com/v4/letter/x/db5fbb/32.png) [@xavi23](https://discuss.elastic.co/u/xavi23)\
**Post date:** [March 28, 2019, 3:32pm UTC](https://discuss.elastic.co/t/dead-letter-queue/174063/5 "2019-03-28T15:32:07Z")

</div>

Hi christian, finally amazon\_e snot support dead\_letter\_quee.

I've many doubts about if I use elasticsearch ouput with dead\_letter\_queue...

The problem tha tI0ve is that when I idnex documetns with logstash I loss some documents, but not is a problem of mapping, I think that is a problem of CPU logstash or something, because in logs of logstash I don't have any error or warning....

If I use elasticsearch liek output, I will see this docuemtns in dead\_letter\_queue or only insert this with errors when index documents ... ?

Another doubt that I've is that my .conf file IO've 23 differents inputs...where is better 1 file with many inputs or 1 input with 1 file.conf .... ?

Thansk foradvanced.

---

<div class="post-metadata">

**Author:** ![xavi23](https://avatars.discourse-cdn.com/v4/letter/x/db5fbb/32.png) [@xavi23](https://discuss.elastic.co/u/xavi23)\
**Post date:** [March 28, 2019, 3:33pm UTC](https://discuss.elastic.co/t/dead-letter-queue/174063/6 "2019-03-28T15:33:31Z")

</div>

And I've another doubt.... It's possible have many logstash and run this logstash sequencly... ?

---

<div class="post-metadata">

**Author:** ![xavi23](https://avatars.discourse-cdn.com/v4/letter/x/db5fbb/32.png) [@xavi23](https://discuss.elastic.co/u/xavi23)\
**Post date:** [April 3, 2019, 7:18am UTC](https://discuss.elastic.co/t/dead-letter-queue/174063/7 "2019-04-03T07:18:48Z")

</div>

Hi, I'm tryin gto use elasticsearch and not use amazon\_es , but I follow to see anything in dead\_letter\_queue ....

I use PERSISTEND queue type and I enable the dead\_letter\_queue : true ...

How I can enable correctly dead\_letter\_queue ... ? I try to index json with mapping error but I can't see in dead\_letter\_queue.

Thanks for advanced.

King regards.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2019, 7:19am UTC](https://discuss.elastic.co/t/dead-letter-queue/174063/8 "2019-05-01T07:19:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
