I was able to write to dead letter queue successfully but despite putting in configuration for re-processing them, I'm unable to see the processed objects in elastic search.
Is there a setting that explicitly tells logstash to send it back again to elastic search?
I have a file called dlq.conf in the /etc/logstash/conf.d/ directory that looks like.
input {
dead_letter_queue {
path => "/var/lib/logstash/dead_letter_queue"
pipeline_id => "main"
commit_offsets => true
}
}
Also my logstash.yml looks like
/etc/logstash/logstash.yml file
config.reload.automatic: true
log.level: info
pipeline:
id: main
workers: 2
path:
config: "/etc/logstash/conf.d/*.conf"
data: "/var/lib/logstash"
logs: /var/log/logstash
queue:
type: persisted
dead_letter_queue.enable: true
path.dead_letter_queue: "/var/lib/logstash/dead_letter_queue"