# Unable to parse logs

**URL:** <https://discuss.elastic.co/t/unable-to-parse-logs/132654>\
**Category:** Logstash\
**Created:** [May 21, 2018, 12:18pm UTC](https://discuss.elastic.co/t/unable-to-parse-logs/132654 "2018-05-21T12:18:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ishantanu](https://avatars.discourse-cdn.com/v4/letter/i/da6949/32.png) [@ishantanu](https://discuss.elastic.co/u/ishantanu)\
**Post date:** [May 21, 2018, 12:18pm UTC](https://discuss.elastic.co/t/unable-to-parse-logs/132654/1 "2018-05-21T12:18:10Z")

</div>

Hi,

I have two different patterns in a single log file and I have two different groks handling them. But, somehow, the second grok is not able to parse the logs. I checked in grok debugger and everything is parsed correctly.

Here's the logstash config:

```auto
  if "devops-logs" in [tags] {
    grok {
      patterns_dir => ["/etc/logstash/conf.d/patterns"]
      match => { "message" => "%{DATE_CUSTOM:date}%{SPACE}%{TIME:time}%{SPACE}%{DATA}%{SPACE}%{LOGLEVEL:log_level}%{SPACE}%{SPACE}%{JOB_NAME}%{SPACE}-%{SPACE}tag%{SPACE}:%{SPACE}%{TAG_NAME:tag},%{SPACE}instance%{SPACE}:%{SPACE}%{NUMBER:instance},%{SPACE}event_date%{SPACE}:%{SPACE}%{DATE_CUSTOM:event_date}%{DATA},%{SPACE}count%{SPACE}:%{SPACE}%{NUMBER:count},%{SPACE}quota_amount%{SPACE}:%{SPACE}%{QUOTA_AMOUNT:quota_amount}" }
      tag_on_failure => ["no-overage-per-quota"]

    }

    grok {
      patterns_dir => ["/etc/logstash/conf.d/patterns"]
      match => { "message" => "%{DATE_CUSTOM:date}%{SPACE}%{TIME:time}%{SPACE}%{DATA}%{SPACE}%{LOGLEVEL:log_level}%{SPACE}%{SPACE}%{JOB_NAME}%{SPACE}-%{SPACE}tag%{SPACE}:%{SPACE}%{TAG_NAME:tag},%{SPACE}instance%{SPACE}:%{SPACE}%{NUMBER:instance},%{SPACE}event_date%{SPACE}:%{SPACE}%{DATE_CUSTOM:event_date}%{DATA},%{SPACE}count%{SPACE}:%{SPACE}%{NUMBER:count},%{SPACE}%{DATA}%{SPACE}:%{SPACE}%{NUMBER:event_id}" }
      remove_tag => ["no-overage-per-quota"]
    }

    mutate {
      convert => { "event_id" => "integer" }
      convert => { "count" => "integer" }
      convert => { "instance" => "integer" }
      convert => { "tag" => "string" }
      convert => { "quota_amount" => "string" }
    }
  }

```

Here's the sample log entry which is not getting parsed:

```auto
2018-05-21 05:00:01.279 [pool-3-thread-1] INFO a.b.c.devops.SqlQueryOutputLoggerJob:34 - tag : events-per-quota, instance : 433974071000110045, event_date : 2018-05-21 00:00:00.0, count : 2142, event_id : 11

```

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 21, 2018, 1:15pm UTC](https://discuss.elastic.co/t/unable-to-parse-logs/132654/2 "2018-05-21T13:15:53Z")

</div>

How do you know it's the second grok that's failing? And why not use a single grok filter that lists both expressions?

Side note: You can convert captured strings to integers already in the grok filter, removing the need for your mutate filter.

---

<div class="post-metadata">

**Author:** ![ishantanu](https://avatars.discourse-cdn.com/v4/letter/i/da6949/32.png) [@ishantanu](https://discuss.elastic.co/u/ishantanu)\
**Post date:** [May 21, 2018, 5:01pm UTC](https://discuss.elastic.co/t/unable-to-parse-logs/132654/3 "2018-05-21T17:01:45Z")

</div>

@magnusbaeck This was a silly mistake on my end. I added `include_lines` on filebeat config which was sending logs matching some pattern only which would match the first grok. This is resolved.

Also, thanks for the suggestions for combining groks and removing mutate filter and declare field-type in the pattern itself.

Just a question:

Is there any performance impact of using single grok with multiple patterns instead of multiple groks with single pattern match?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 21, 2018, 6:50pm UTC](https://discuss.elastic.co/t/unable-to-parse-logs/132654/4 "2018-05-21T18:50:57Z")

</div>

> Is there any performance impact of using single grok with multiple patterns instead of multiple groks with single pattern match?

If any you should see a slight gain in having a single filter, but if it's that important to you you should measure it yourself.

---

<div class="post-metadata">

**Author:** ![ishantanu](https://avatars.discourse-cdn.com/v4/letter/i/da6949/32.png) [@ishantanu](https://discuss.elastic.co/u/ishantanu)\
**Post date:** [May 22, 2018, 4:50am UTC](https://discuss.elastic.co/t/unable-to-parse-logs/132654/5 "2018-05-22T04:50:11Z")

</div>

Okay. Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 19, 2018, 4:50am UTC](https://discuss.elastic.co/t/unable-to-parse-logs/132654/6 "2018-06-19T04:50:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.