Grok parse failures

Elastic Stack Logstash
1

Hello I'm playing around with grok filters and i'm running into parse failures..Any idea whats wrong ?

Filters:

filter {
  if [type] == "app-data" {
    mutate {
      rename => ["env", "environment"]
    }
    grok {
      break_on_match => false
      match => {
      "message" => "^%{DATA:timestamp_local}\|%{DATA:log_level}\|%{DATA:ID}\|%{WORD:Type}\|%{WORD:stage}\|%{NUMBER:accountNumber}\|%{WORD:region}"
      }
    }
  }
}

Here's the input being provided:

2020-12-01T10:28:51.603Z|INFO|AP92|com.test.resource.6|preview-5|9244208|US-EAST-9

Here's the error i see:

{
           "tags" => [
        [0] "_grokparsefailure"
    ],
           "type" => "app-data",
       "hostName" => "ELB-1",
       "@version" => "1",
           "path" => "/Users/metrics-poc/filebeat-output.log",
           "host" => "MA81",
     "@timestamp" => 2020-12-01T15:28:52.967Z,
        "message" => "2020-12-01T10:28:51.603Z|INFO|AP92|com.test.resource.6|preview-5|9244208|US-EAST-9",
    "environment" => "prod"
}
2

WORD will not match either of those fields. WORD match word characters, which are [a-zA-Z0-9_].

3

Gotcha..What difference does it makes if we generalize the type to DATA which seems like applicable to almost everything?

4

Sometimes DATA will not do what you expect. You might be better off with a custom pattern

<?(Type)[^|]*>

will match anything that is not a pipe character.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.