# Messages grok parse failure

**URL:** <https://discuss.elastic.co/t/messages-grok-parse-failure/91806>\
**Category:** Logstash\
**Created:** [July 4, 2017, 7:23pm UTC](https://discuss.elastic.co/t/messages-grok-parse-failure/91806 "2017-07-04T19:23:10Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mvelez83](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@mvelez83](https://discuss.elastic.co/u/mvelez83)\
**Post date:** [July 4, 2017, 7:23pm UTC](https://discuss.elastic.co/t/messages-grok-parse-failure/91806/1 "2017-07-04T19:23:10Z")

</div>

Hello,

I'm very new to creating Grok filters and I was going to create one for /var/log/messages (For named). Here is my logstash configuration I attempted to use:

> input {  
> redis {  
> host =\> "[redis01.nyc.aevtech.net](http://redis01.nyc.aevtech.net)"  
> data\_type =\> "list"  
> key =\> "dnsint"  
> password =\> "qwarsdw5243"  
> }  
> }  
> filter {  
> grok {  
> match =\> { "message" =\> "%{SYSLOGBASE} %{GREEDYDATA}" }  
> }  
> }  
> output {  
> elasticsearch { hosts =\> ["[es01.nyc.aevtech.net:9200](http://es01.nyc.aevtech.net:9200)"]  
> index =\> "logstash-%{+YYYY.MM.dd}" }  
> }

I can see the logs coming in but I also see the following tags with each log entry coming in for /var/log/messages:

> tags: \_grokparsefailure, \_geoip\_lookup\_failure, \_dateparsefailure

I'm not 100% sure what the issue is. Also, I tried using [Automatic grok discovery](http://grokconstructor.appspot.com/do/automatic) to come up with this grok filter but as mentioned above this is very new to me. Thank you for any and all help in advanced and let me know if anything else is needed to help assist so I can send it as soon as possible.

Thank You

---

<div class="post-metadata">

**Author:** ![mvelez83](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@mvelez83](https://discuss.elastic.co/u/mvelez83)\
**Post date:** [July 5, 2017, 1:36am UTC](https://discuss.elastic.co/t/messages-grok-parse-failure/91806/2 "2017-07-05T01:36:29Z")

</div>

Hello,

Here is some of the log entries I would like this grok filter to work for:

> Jul 4 13:14:09 dns01 named[31894]: validating @0x7fadc0710810: [sumo.external.zlb.scl3.mozilla.com](http://sumo.external.zlb.scl3.mozilla.com) A: no valid signature found  
> Jul 4 13:59:39 dns01 named[31894]: error (connection refused) resolving '[cdn-hls.movies.com.fpbns.net/A/IN](http://cdn-hls.movies.com.fpbns.net/A/IN)': 192.20.123.10#53  
> Jul 4 14:39:15 dns01 named[31894]: dispatch 0x7fadc00f5f20: open\_socket(0.0.0.0#8614) -\> permission denied: continuing  
> Jul 4 15:27:09 dns01 named[31894]: validating @0x7fadc0710810: [choices.truste.com](http://choices.truste.com) A: no valid signature found  
> Jul 4 15:28:10 dns01 named[31894]: validating @0x7fadc067b940: [choices-or.truste.com](http://choices-or.truste.com) A: no valid signature found  
> Jul 4 15:38:39 dns01 named[31894]: validating @0x7fadc067b940: [choices.truste.com](http://choices.truste.com) A: no valid signature found  
> Jul 4 15:38:42 dns01 named[31894]: validating @0x7fadc0710810: [choices-or.truste.com](http://choices-or.truste.com) A: no valid signature found  
> Jul 4 17:42:18 dns01 named[31894]: dispatch 0x7fadc00f6540: open\_socket(0.0.0.0#4321) -\> permission denied: continuing  
> Jul 4 17:42:37 dns01 named[31894]: error (connection refused) resolving '[webdir.online.lync.com/A/IN](http://webdir.online.lync.com/A/IN)': 8.26.204.25#53  
> Jul 4 19:01:15 dns01 named[31894]: dispatch 0x7fadc00f6b60: open\_socket(0.0.0.0#8610) -\> permission denied: continuing

---

<div class="post-metadata">

**Author:** ![Nico-DF](https://avatars.discourse-cdn.com/v4/letter/n/ed8c4c/32.png) [@Nico-DF](https://discuss.elastic.co/u/Nico-DF)\
**Post date:** [July 5, 2017, 7:30am UTC](https://discuss.elastic.co/t/messages-grok-parse-failure/91806/3 "2017-07-05T07:30:09Z")

</div>

You're currently using SYSLOGBASE in your pattern.

If you check what it is ([http://grokdebug.herokuapp.com/patterns#](http://grokdebug.herokuapp.com/patterns#)), you can find the following:

```auto
SYSLOGBASE %{SYSLOGTIMESTAMP:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}:

SYSLOGTIMESTAMP %{MONTH} +%{MONTHDAY} %{TIME}
PROG (?:[\w._/%-]+)
SYSLOGPROG %{PROG:program}(?:\[%{POSINT:pid}\])?
SYSLOGHOST %{IPORHOST}
SYSLOGFACILITY <%{NONNEGINT:facility}.%{NONNEGINT:priority}>

```

So basically, you see that your log line does not match this pattern. Hence the grokparsefailure

---

<div class="post-metadata">

**Author:** ![mvelez83](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@mvelez83](https://discuss.elastic.co/u/mvelez83)\
**Post date:** [July 5, 2017, 9:33am UTC](https://discuss.elastic.co/t/messages-grok-parse-failure/91806/4 "2017-07-05T09:33:58Z")

</div>

Hello,

Thank you very much for your reply. So, based off of what you said I tried a new one and also passed it along the Grok Debugger and came up with the following:

> %{SYSLOGBASE2} %{GREEDYDATA}

This produced this:

> {  
> "SYSLOGBASE2": [  
> [  
> "Jul 5 04:54:18 dns01 systemd:"  
> ]  
> ],  
> "timestamp": [  
> [  
> "Jul 5 04:54:18"  
> ]  
> ],  
> "MONTH": [  
> [  
> "Jul"  
> ]  
> ],  
> "MONTHDAY": [  
> [  
> "5",  
> null  
> ]  
> ],  
> "TIME": [  
> [  
> "04:54:18"  
> ]  
> ],  
> "HOUR": [  
> [  
> "04",  
> null,  
> null  
> ]  
> ],  
> "MINUTE": [  
> [  
> "54",  
> null,  
> null  
> ]  
> ],  
> "SECOND": [  
> [  
> "18",  
> null  
> ]  
> ],  
> "timestamp8601": [  
> [  
> null  
> ]  
> ],  
> "YEAR": [  
> [  
> null  
> ]  
> ],  
> "MONTHNUM": [  
> [  
> null  
> ]  
> ],  
> "ISO8601\_TIMEZONE": [  
> [  
> null  
> ]  
> ],  
> "SYSLOGFACILITY": [  
> [  
> null  
> ]  
> ],  
> "facility": [  
> [  
> null  
> ]  
> ],  
> "priority": [  
> [  
> null  
> ]  
> ],  
> "logsource": [  
> [  
> "dns01"  
> ]  
> ],  
> "IPORHOST": [  
> [  
> "dns01"  
> ]  
> ],  
> "HOSTNAME": [  
> [  
> "dns01"  
> ]  
> ],  
> "IP": [  
> [  
> null  
> ]  
> ],  
> "IPV6": [  
> [  
> null  
> ]  
> ],  
> "IPV4": [  
> [  
> null  
> ]  
> ],  
> "SYSLOGPROG": [  
> [  
> "systemd"  
> ]  
> ],  
> "program": [  
> [  
> "systemd"  
> ]  
> ],  
> "pid": [  
> [  
> null  
> ]  
> ],  
> "GREEDYDATA": [  
> [  
> "Starting Session 313 of user mvelez."  
> ]  
> ]  
> }

When I restarted logstash I can see the logs coming in with more fields and options compared to before but I'm still seeing the tags with the parse failures. Is it because it is not grabbing the full line in the pattern? Compared to before, I can grab much more data from the logs. I guess the question for me now is that although the above is a bit better will those tags be an issue?

Thank You!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2017, 9:34am UTC](https://discuss.elastic.co/t/messages-grok-parse-failure/91806/5 "2017-08-02T09:34:07Z")

</div>

Closed on August 2, 2017, 9:34am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.