# \_grokparsefailure Tag when parsing logs via logstash **URL:** \ **Category:** Logstash\ **Created:** [April 4, 2016, 10:22am UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-when-parsing-logs-via-logstash/46224 "2016-04-04T10:22:16Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![aviral\_srivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aviral_srivastava/32/98018_2.png) [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Post date:** [April 4, 2016, 10:22am UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-when-parsing-logs-via-logstash/46224/1 "2016-04-04T10:22:16Z")
**I have the following log:-** 2016-03-31 12:40:40 INFO SmartAppUtils:981 - Organization: AppsTwo 2016-03-31 12:40:40 INFO SmartAppUtils:988 - Brand organization: AppsTwo 2016-03-31 12:40:51 INFO SmartAppUtils:981 - Organization: AppsTwo 2016-03-31 12:40:51 INFO SmartAppUtils:988 - Brand organization: AppsTwo **My logstash configuration file as follows:-** input { file { type =\> "tomcat" path =\> ["D:/logs/Smart\_logs/smartlogstest.log"] codec =\> multiline { negate =\> true pattern =\> "(^%{URIHOST} %{HAPROXYTIME})" what =\> "previous" } } } filter { ``` if [type] == "tomcat" { grok{ patterns_dir => "./patterns" match => ["message", "%{SMART_TIMESTAMP:timestamp} %{LOGLEVEL: logLevel}:%{GREEDYDATA:message}"] overwrite => ["message"] } date{ match=>["timestamp","yyyy-MM-dd HH:mm:ss"] } } ``` } output { stdout { codec=\>rubydebug } elasticsearch{ hosts=\>"localhost" index=\>"smartlogs\_test" } } **The pattern under multiline codec is created using grok debugger,** **SMART\_TIMESTAMP is a custom pattern which I have defined under patterns folder in a file named extra.conf** **as follows:-** **SMART\_TIMESTAMP (%{URIHOST} %{HAPROXYTIME})** It seems that pattern is not matching but I have checked using grok debugger. URIHOST returns 2016-03-31. and HAPROXYTIME returns 12:40:40 ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 4, 2016, 10:28am UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-when-parsing-logs-via-logstash/46224/2 "2016-04-04T10:28:47Z")
Comments: - Your definition of SMART\_TIMESTAMP doesn't make sense. URIHOST has nothing to do with a yyyy-mm-dd date. Why not use TIMESTAMP\_ISO8601 instead? - Remove the space between "LOGLEVEL:" and "logLevel". - In your grok expression you have a colon after the loglevel but there's no colon there in the actual log message. ---
**Author:** ![aviral\_srivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aviral_srivastava/32/98018_2.png) [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Post date:** [April 4, 2016, 1:08pm UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-when-parsing-logs-via-logstash/46224/3 "2016-04-04T13:08:33Z")
(post withdrawn by author, will be automatically deleted in 24 hours unless flagged) ---
**Author:** ![aviral\_srivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aviral_srivastava/32/98018_2.png) [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Post date:** [April 4, 2016, 1:44pm UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-when-parsing-logs-via-logstash/46224/5 "2016-04-04T13:44:26Z")
Thanks, **magnusbaeck** the logs are now started getting parsed by logstash. Could you please tell me that how we decide that we need to use **TIMESTAMP\_ISO8601**. When I use grokdebugger why it didn't generated the pattern TIMESTAMP\_ISO8601. May be it's a silly question but your answer will clear my doubts. **Below is my conf:-** input { file { type =\> "tomcat" path =\> ["D:/logs/Smart\_logs/smartlogstest.log"] codec =\> multiline { negate =\> true pattern =\> "(^%{TIMESTAMP\_ISO8601})" what =\> "previous" } } } filter { if [type] == "tomcat" { ``` grok{ match => ["message", "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:logLevel} %{NOTSPACE:className}:%{NUMBER:line} - %{GREEDYDATA:message}"] overwrite => ["message"] } date{ match=>["timestamp","yyyy-MM-dd HH:mm:ss"] } ``` } } output { stdout { codec=\>rubydebug } elasticsearch{ hosts=\>"localhost" index=\>"smartlogs\_test" } } **I have these log lines in the same log file as follows:-** 2016-03-31 13:00:05 ERROR FlashSummarySubBuMismatchReportListener:47 - Scheduler failed ... Logging the error stack ... org.hibernate.exception.SQLGrammarException: could not execute query 2016-03-31 13:00:05 ERROR FlashSummarySubBuMismatchReportListener:48 - Scheduled job running failed ...org.hibernate.exception.SQLGrammarException: could not execute query 2016-03-31 13:00:07 ERROR EmailFactory:537 - Sending error information failed 2016-03-31 13:00:07 ERROR FlashSummaryLessFTEReportListener:54 - Scheduler failed ... Logging the error stack ... org.hibernate.exception.SQLGrammarException: could not execute query **They are not getting parsed. The pattern worked with earlier logs but not with these.** ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [April 4, 2016, 5:03pm UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-when-parsing-logs-via-logstash/46224/6 "2016-04-04T17:03:48Z")
> Could you please tell me that how we decide that we need to use TIMESTAMP\_ISO8601. When I use grokdebugger why it didn't generated the pattern TIMESTAMP\_ISO8601. You mean grokconstructor? The grokdebugger site is only for testing your existing expression, right? There can be many grok patterns that match a particular input string and a program doesn't always have the necessary context to make the right decision. Only you know what kind of data it is and what expression describes it best. > They are not getting parsed. The pattern worked with earlier logs but not with these. In your expression you have two spaces between the LOGLEVEL and NOTSPACE patterns but your actual log doesn't have that. ---
**Author:** ![aviral\_srivastava](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aviral_srivastava/32/98018_2.png) [@aviral\_srivastava](https://discuss.elastic.co/u/aviral_srivastava)\ **Post date:** [April 6, 2016, 11:03am UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-when-parsing-logs-via-logstash/46224/7 "2016-04-06T11:03:12Z")
Thanks, for the reply, that was a silly mistake on my end. The log:- **2016-03-31 12:40:51 INFO SmartAppUtils:988 - Brand organization: AppsTwo** **2016-03-31 13:00:00 WARN FlashSummarySubBuMismatchReportListener:34 - Running FlashSummarySubBuMismatchReportListener ...** had two whitespaces between LOGLEVEL and NOTSPACE but the log:- **2016-03-31 13:00:05 ERROR FlashSummaryReportPrjBlankMessageListener:53 - Scheduler failed ... Logging the error stack ... org.hibernate.exception.SQLGrammarException: could not execute query** had one space between LOGLEVEL and NOTSPACE. Because of this my expression was not working:- **%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:logLevel} %{NOTSPACE:className}:%{NUMBER:line} - %{GREEDYDATA:message}** as I assumed two whitespaces between LOGLEVEL and NOTSPACE. Below is my corrected pattern:- **%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:logLevel}%{SPACE}%{NOTSPACE:className}:%{NUMBER:line} - %{GREEDYDATA:message}** ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:03am UTC](https://discuss.elastic.co/t/-grokparsefailure-tag-when-parsing-logs-via-logstash/46224/8 "2017-07-06T05:03:32Z")