# Grok parse failed, works good with grok debugger **URL:** \ **Category:** Logstash\ **Created:** [October 26, 2017, 7:27pm UTC](https://discuss.elastic.co/t/grok-parse-failed-works-good-with-grok-debugger/105473 "2017-10-26T19:27:48Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![sarikonl](https://avatars.discourse-cdn.com/v4/letter/s/bbe5ce/32.png) [@sarikonl](https://discuss.elastic.co/u/sarikonl)\ **Post date:** [October 26, 2017, 7:27pm UTC](https://discuss.elastic.co/t/grok-parse-failed-works-good-with-grok-debugger/105473/1 "2017-10-26T19:27:48Z")
Can someone please help me with this. I tried almost all possible ways but no luck. My grok debugger works perfect but get "beats\_input\_codec\_plain\_applied, failedparse" error in Kibana from logstash. # This is the grok I am using: input { beats { port =\> 5054 } } filter { grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:logdate}}"} # if we can't parse the log message then add a tag to it to identify it tag\_on\_failure =\> ["failedparse"] } # date match on the date of the log entry using a custom format match, not specifying a target so this date result will go into the @timestamp field date { match =\> ["logdate", "YYYY-MM-dd HH:mm:ss,SSS"] } # date match on the date of the log entry using a custom format match, specify the target to put this date result in the logdate field date { match =\> ["logdate", "YYYY-MM-dd HH:mm:ss,SSS"] target =\> "logdate" } } output { elasticsearch { # output to elasticsearh, use templates to parse the data and save in an index noliouat-request then append the year and month hosts =\> ["localhost:9200"] manage\_template =\> true index =\> "uat-dmall-%{+[YYYY.MM](http://YYYY.MM)}" } } ============================================================= Sample Log file: 2017-10-26 19:17:59,894 [org.springframework.jms.listener.DefaultMessageListenerContainer#3-1] INFO (com.app.platform.server.dataservices.services.execmng.ConnectivityUpdaterImpl:92) - Done updating connectivity: es\_hostname 2017-10-26 19:18:04,216 [ExecutionServerStatusTask-41322] DEBUG (com.app.platform.server.dataservices.services.execmng.ExecutionServerStatusMonitor:92) - Update execution servers status 2017-10-26 19:17:53,633 [JobDisposer-1] INFO (com.app.platform.server.dataservices.services.execution.ExecutionServiceImpl:1262) - disposing job [2162707]. Appreciate a quick turn around. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 27, 2017, 3:38am UTC](https://discuss.elastic.co/t/grok-parse-failed-works-good-with-grok-debugger/105473/2 "2017-10-27T03:38:21Z")
There are two `}` in your grok expression. ---
**Author:** ![sarikonl](https://avatars.discourse-cdn.com/v4/letter/s/bbe5ce/32.png) [@sarikonl](https://discuss.elastic.co/u/sarikonl)\ **Post date:** [October 27, 2017, 2:00pm UTC](https://discuss.elastic.co/t/grok-parse-failed-works-good-with-grok-debugger/105473/3 "2017-10-27T14:00:12Z")
I see there are the equal number of opening and closing { }. Can you advise what I am overlooking? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 27, 2017, 2:04pm UTC](https://discuss.elastic.co/t/grok-parse-failed-works-good-with-grok-debugger/105473/4 "2017-10-27T14:04:39Z")
The grok expression, i.e. the string between the double quotes, is `%{TIMESTAMP_ISO8601:logdate}}`. That's one `{` and two `}`: ---
**Author:** ![sarikonl](https://avatars.discourse-cdn.com/v4/letter/s/bbe5ce/32.png) [@sarikonl](https://discuss.elastic.co/u/sarikonl)\ **Post date:** [October 30, 2017, 3:21pm UTC](https://discuss.elastic.co/t/grok-parse-failed-works-good-with-grok-debugger/105473/5 "2017-10-30T15:21:42Z")
Thanks..figured that out. But I still have Parse failed error from my grok... Can you please suggest the best grok pattern to handle this.... ======================================================================= input { beats { port =\> 5054 } } filter { grok { match =\> { "message" =\> "%{TIMESTAMP\_ISO8601:tempdate} [%{NOTSPACE:thread} %{WORD:type}%{SPACE}(%{JAVACLASS:program}:%{NUMBER:programline}) - %{GREEDYDATA:logmessage}"} # if we can't parse the log message then add a tag to it to identify it tag\_on\_failure =\> ["failedparse"] } mutate { split =\> { "tempadate" =\> "," } add\_field =\> { "date" =\> "%{tempdate[0]}" } remove\_field =\> ["tempdate"] } # date match on the date of the log entry using a custom format match, not specifying a target so this date result will go into the @timestamp field date { match =\> ["date", "YYYY-MM-dd HH:mm:ss"] } } ====================================================================================== Sample log file: 2017-10-30 14:34:13,362 [JobDisposer-1] INFO (com.platform.server.dataservices.services.execution.ExecutionServiceImpl:1262) - disposing job [1441868]. 2017-10-30 14:34:45,958 [ExecutionServerStatusTask-13983] DEBUG (com.platform.server.dataservices.services.execmng.ExecutionServerStatusMonitor:92) - Update servers status 2017-10-30 14:34:46,514 [Finalizer] DEBUG (com.platform.connection.PoolingClientConnectionManager:275) - Connection manager is shutting down 2017-10-30 14:34:48,703 [eventsProcessingHandler[PerJobExecutorsManager\_garbageCollector]-1] DEBUG (com..platform.server.dataservices.services.executors.PerJobExecutorsManagerImpl:232) - Shutting down executors for finished jobs and cleaning up resources... 2017-10-30 14:34:48,703 [AgentHttpRequestsHandler-[PerJobExecutorsManager\_garbageCollector]-1] DEBUG (com.platform.server.dataservices.services.executors.PerJobExecutorsManagerImpl:232) - Shutting down executors for finished jobs and cleaning up resources... 2017-10-30 14:34:48,713 [org.springframework.jms.listener.DefaultMessageListenerContainer#3-1] INFO (com.platform.server.dataservices.services.execmng.ConnectivityUpdaterImpl:83) - Updating connectivity. current details: Execut ================================================================== The issue I think is that the parse is failing while handling the major difference in between lines with [.......] and [...[....]] Please advise on how this can be handled. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 30, 2017, 8:01pm UTC](https://discuss.elastic.co/t/grok-parse-failed-works-good-with-grok-debugger/105473/6 "2017-10-30T20:01:54Z")
Please post your configuration as preformatted text so that backslashes don't get stripped. > [%{NOTSPACE:thread} Where's the closing square bracket? > mutate { > split =\> { "tempadate" =\> "," } > add\_field =\> { "date" =\> "%{tempdate[0]}" } > remove\_field =\> ["tempdate"] > } Why on earth are you doing this? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 27, 2017, 8:01pm UTC](https://discuss.elastic.co/t/grok-parse-failed-works-good-with-grok-debugger/105473/7 "2017-11-27T20:01:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.