Can you apply another grok filter in case of _grokparsefailure?

Elastic Stack Logstash
1

Hey,
I'm working with different log structure and l Tried to adapt my grok filter for most of logs, but there still some that cause a grokparsefailure.

I wonder what is the best way to handle a grokparsefailure?

In my case I have something like this:

filter {

grok {
      match => { message => "(%{TIMESTAMP_ISO8601:timestamp_tmp}\s+\[%{DATA:task}\]\s+%{LOGLEVEL:loglevel}\s+\[%{DATA:package}\]((\s{2}(?m)(?<msgdetail>.*))|(.*\"\s+(?m)(?<msgdtail>.*))|(\+(?m)(?<msgdetail>.*)|.*]\s+(?m)(?<msgdetail>.*))))|(\A%{TIMESTAMP_ISO8601:timestamp_tmp}\s+(.*)%{LOGLEVEL:loglevel}\s+\[%{DATA:package}\]\s+(?m)(?<msgdetail>.*)) | (%{TIMESTAMP_ISO8601:timestamp_tmp}\s+%{LOGLEVEL:loglevel} (?m)(?<msgdetail>.*))"  }


    }



  if "_grokparsefailure" in [tags] {
        grok {
            match => {message => "%{TIMESTAMP_ISO8601:timestamp_tmp} (?<task>\[.*\]) %{LOGLEVEL:loglevel} (?m)(?<masgdetail>.*)"}
        }
    }


  mutate{

      rename => { "message" => "raw_message" }
      rename => { "msgdetail" => "message" }
      remove_field => "task"
      lowercase => ["loglevel"]
}

}

I have huge amounts of logs and I couldn't manage to test the if statement part.
Is this gonna work? or is there a better approach to do this?

2

A more compact way to do that would be

grok {
    match => {
        message => [
            "(%{TIMESTAMP_ISO8601:timestamp_tmp}\s+\[%{DATA:task}\]\s+%{LOGLEVEL:loglevel}\s+\[%{DATA:package}\]((\s{2}(?m)(?<msgdetail>.*))|(.*\"\s+(?m)(?<msgdtail>.*))|(\+(?m)(?<msgdetail>.*)|.*]\s+(?m)(?<msgdetail>.*))))|(\A%{TIMESTAMP_ISO8601:timestamp_tmp}\s+(.*)%{LOGLEVEL:loglevel}\s+\[%{DATA:package}\]\s+(?m)(?<msgdetail>.*)) | (%{TIMESTAMP_ISO8601:timestamp_tmp}\s+%{LOGLEVEL:loglevel} (?m)(?<msgdetail>.*))",
            "%{TIMESTAMP_ISO8601:timestamp_tmp} (?<task>\[.*\]) %{LOGLEVEL:loglevel} (?m)(?<masgdetail>.*)"
    }
}

You should probably anchor those patterns to start of line using ^ at the beginning.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.