[WARN ][org.logstash.common.io.DeadLetterQueueWriter] Event previously submitted to dead letter queue. Skipping


#1

I just started working with the Dead Letter Queue, but after restarting my pipelines a few times I am now getting this message repeatedly in my logs:

[WARN ][org.logstash.common.io.DeadLetterQueueWriter] Event previously submitted to dead letter queue. Skipping...

I tried deleting all the files in the dead_letter_queue directories, but that doesn't seem to get it to start from the beginning again. Any help would be appreciated.

@warkolm any thoughts on this? sorry to mention you, i just really need help with this one and i feel like i've tried everything and read all the docs.

Edit: Looks like the issue was that I was actually clearing the queue, but the warning was because the DLQ pipeline was not resolving the mapping issues for the documents. When I read the warning log message I thought it was coming from the DLQ pipeline the first timeit tried to process the event.

It would be awesome if it were more verbose. All I saw were Dead Letter Queue warnings. I didn't see any of the usual "Elasticsearch couldn't index..." warnings. I'm not sure why those weren't included. Does DLQ being turned on suppress those warnings or something?


#2

I'm also seeing later on that the DLQ max size limit has been reached. How can I work around this? Is the size limit for each file (1.log, 2.log, etc.) or for the entire pipeline directory?


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.