Logstash deadletterqueue

[fadihaddad (fadi)]

i enabled deadletterqueue in logtsash.yml and created the pipeline deadletterqueue in pipelines.yml wih id dlq
so this os my config
input {
dead_letter_queue {
path => "/var/lib/logstash/dead_letter_queue"
commit_offsets => true
pipeline_id => "dlq"
}
}

output {
elasticsearch {
hosts => "hostip"
index => "deadletterque"
document_type => "data"

}
stdout {
codec => rubydebug { metadata => true }
}
}
the pipe is running but stuck in flushing loop ts seems its not finding the file path to index i used the rpm installation and i accessed the deadletter queue in /var/lib/logstash and fund the piplines and logs inside them what can i do to fix this

[Badger]

@fadihaddad

input {
dead_letter_queue {
path => "/var/lib/logstash/dead_letter_queue"
commit_offsets => true
pipeline_id => "dlq"
}
}

That says you are trying to consume the failed messages of an elasticsearch output in a pipline called dlq. Is that really what you want? pipeline_id refers to the id of the pipeline that is adding messages to the dlq, it is not a way to name the dlq consuming pipeline.

What is the name of the pipeline that is failing to write messages to an elasticsearch output, that is the name you should be using here.

[fadihaddad (fadi)]

maybe I want to use it on multiple pipelines?

[Badger]

Then I believe you would need multiple consumer pipelines, since pipeline_id is a string, not an array.