Unable to parse logs

[ishantanu (Shantanu Deshpande)]

Hi,

I have two different patterns in a single log file and I have two different groks handling them. But, somehow, the second grok is not able to parse the logs. I checked in grok debugger and everything is parsed correctly.

Here's the logstash config:

  if "devops-logs" in [tags] {
    grok {
      patterns_dir => ["/etc/logstash/conf.d/patterns"]
      match => { "message" => "%{DATE_CUSTOM:date}%{SPACE}%{TIME:time}%{SPACE}%{DATA}%{SPACE}%{LOGLEVEL:log_level}%{SPACE}%{SPACE}%{JOB_NAME}%{SPACE}-%{SPACE}tag%{SPACE}:%{SPACE}%{TAG_NAME:tag},%{SPACE}instance%{SPACE}:%{SPACE}%{NUMBER:instance},%{SPACE}event_date%{SPACE}:%{SPACE}%{DATE_CUSTOM:event_date}%{DATA},%{SPACE}count%{SPACE}:%{SPACE}%{NUMBER:count},%{SPACE}quota_amount%{SPACE}:%{SPACE}%{QUOTA_AMOUNT:quota_amount}" }
      tag_on_failure => ["no-overage-per-quota"]

    }

    grok {
      patterns_dir => ["/etc/logstash/conf.d/patterns"]
      match => { "message" => "%{DATE_CUSTOM:date}%{SPACE}%{TIME:time}%{SPACE}%{DATA}%{SPACE}%{LOGLEVEL:log_level}%{SPACE}%{SPACE}%{JOB_NAME}%{SPACE}-%{SPACE}tag%{SPACE}:%{SPACE}%{TAG_NAME:tag},%{SPACE}instance%{SPACE}:%{SPACE}%{NUMBER:instance},%{SPACE}event_date%{SPACE}:%{SPACE}%{DATE_CUSTOM:event_date}%{DATA},%{SPACE}count%{SPACE}:%{SPACE}%{NUMBER:count},%{SPACE}%{DATA}%{SPACE}:%{SPACE}%{NUMBER:event_id}" }
      remove_tag => ["no-overage-per-quota"]
    }

    mutate {
      convert => { "event_id" => "integer" }
      convert => { "count" => "integer" }
      convert => { "instance" => "integer" }
      convert => { "tag" => "string" }
      convert => { "quota_amount" => "string" }
    }
  }

Here's the sample log entry which is not getting parsed:

2018-05-21 05:00:01.279 [pool-3-thread-1] INFO  a.b.c.devops.SqlQueryOutputLoggerJob:34 - tag : events-per-quota, instance : 433974071000110045, event_date : 2018-05-21 00:00:00.0, count : 2142, event_id : 11

[magnusbaeck (Magnus Bäck)]

How do you know it's the second grok that's failing? And why not use a single grok filter that lists both expressions?

Side note: You can convert captured strings to integers already in the grok filter, removing the need for your mutate filter.

[ishantanu (Shantanu Deshpande)]

@magnusbaeck This was a silly mistake on my end. I added include_lines on filebeat config which was sending logs matching some pattern only which would match the first grok. This is resolved.

Also, thanks for the suggestions for combining groks and removing mutate filter and declare field-type in the pattern itself.

Just a question:

Is there any performance impact of using single grok with multiple patterns instead of multiple groks with single pattern match?

[magnusbaeck (Magnus Bäck)]

Is there any performance impact of using single grok with multiple patterns instead of multiple groks with single pattern match?

If any you should see a slight gain in having a single filter, but if it's that important to you you should measure it yourself.

[ishantanu (Shantanu Deshpande)]

Okay. Thanks.