Hi there, we've recently started looking into setting up a
dead_letter_queue pipeline at Discourse but I had some questions which wasn't immediately clear from the documentation.
Does the DLQ log files get deleted at all? From what I'm seeing currently, the log files remain on disk even after it has been processed by a pipeline. Also, when does logstash decide to create a new DLQ log file? (1.log, 2.log etc...)
What is the recommended practice for setting up a
dead_letter_queuepipeline? Since the DQL log files are not really human readable, is it recommend to setup the pipeline containing by a
Is it recommended to set
commit_offsetto true for monitoring purposes and then flick it to false once you're ready to fix the underlying bug?
true, where does logstash save the state?
Thank you in advance!