I configure DLQ queue for my pipeline. It will create <date>-dlq
index in elasticsearch. I can see my documents but I would like to see the problem that cause that the document end in DLQ. According to this article, there is @metadata
that include the reason
.
I want to add field where will be this information, but I still receive error. This is my filter of dlq pipeline:
filter {
mutate {
add_field => [ "reason" => "%{[@metadata][dead_letter_queue][reason]}" ]
}
}
Logstas now generate logs like this:
[2018-09-18T18:01:59,410][ERROR][logstash.agent ] Failed to execute action {:id=>:"dlq-pipeline", :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Expected one of #, {, ,, ] at line 10, column 29 (byte 219) after filter {\n mutate {\n add_field => [ \"reason\" ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:42:in
compile_imperative'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:50:in compile_graph'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:12:in
block in compile_sources'", "org/jruby/RubyArray.java:2486:in map'", "/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:11:in
compile_sources'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:51:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:34:in
execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in
with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in block in converge_state'", "org/jruby/RubyArray.java:1734:in
each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in
block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in
converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:105:in block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/interval.rb:18:in
interval'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:94:in execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in
block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in block in initialize'"]}
Is there a way, how to get all @metadata
, or at least the reason
?