Dissect filter plugin to map nested fields

Elastic Stack Logstash
1

I am listening to events like this with logstash:

{"MYSQL":{"eqp":"ST_22","eqpId":"TT_1","label":"MONITOR : BW_TE_TRACKING [20]","local":"2020-09-23T22:35:29.855+05:00"},"tags":["MYSQL"]}

My goal is to store it in MongoDB. So that, output {} is already configured and working but I want to make some filters in the events in the following way:

filter {
  json {
    source => "message"
  }
  mutate {
    remove_field => ["message"]
  }
  if [MYSQL][label] =~ /^.+\[\d+\]$/ {
    dissect {
      mapping => {
        "MYSQL.label" => "%{system}:%{function} [%{msg_id}]"
      }
      add_field => { "XMLnumber" => "%{msg_id}" }
    }
  }
}

My issue starts in the mapping section because I want to map a nested field and I have an ERROR log message from Logstash as if MYSQL.label field didn't exist:

[2020-10-07T18:06:08,521][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Expected one of [ \\t\\r\\n], \"#\", \"=>\" at line 22, column 18 (byte 406) after filter {\r\n  json {\r\n    source => \"message\"\r\n  }\r\n  mutate {\r\n    remove_field => [\"message\"]\r\n  }\r\n  if [MYSQL][label] =~ /^.+\\[\\d+\\]$/ {\r\n    dissect {\r\n      mapping=> {\r\n        \"MYSQL\"", :backtrace=>["C:/logstash-7.9.2/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:183:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "C:/logstash-7.9.2/logstash-core/lib/logstash/java_pipeline.rb:44:in `initialize'", "C:/logstash-7.9.2/logstash-core/lib/logstash/pipeline_action/create.rb:52:in `execute'", "C:/logstash-7.9.2/logstash-core/lib/logstash/agent.rb:357:in `block in converge_state'"]}

It seems it is not the way to call a nested field in Logstash config file. Any help would be appreciated.

Thank you

2

That should be

"[MYSQL][label]" => "%{system}:%{function} [%{msg_id}]"

But the error message says ...dissect {\r\n mapping=> {\r\n \"MYSQL\"", so there is a double quote immediately following MYSQL in your configuration file. You are not running the configuration you think you are.

3

Thank you for your quick response @Badger . It works! I just tried before with brackets but I forgot the quotes. :+1:

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.