# LogStash config not parsing out log events even though config checks out in Grok Debugger **URL:** \ **Category:** Logstash\ **Created:** [August 10, 2017, 5:57pm UTC](https://discuss.elastic.co/t/logstash-config-not-parsing-out-log-events-even-though-config-checks-out-in-grok-debugger/96653 "2017-08-10T17:57:35Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Post date:** [August 10, 2017, 5:57pm UTC](https://discuss.elastic.co/t/logstash-config-not-parsing-out-log-events-even-though-config-checks-out-in-grok-debugger/96653/1 "2017-08-10T17:57:36Z")
First post and hoping this is the forum to post questions ... I have the following GROK filter: if ([service] == "MY\_SERVICE"){ if([attributes][file\_path] == "/apps/log/mydoc/wso2carbon.log") { grok { patterns\_dir =\> ["/apps/logstash-patterns"] match =\> ["body", "%{wso2carbon}"] } kv { source =\> "kvpairs" field\_split =\> ", " value\_split =\> " = " remove\_field =\> ["kvpairs"] } date { match =\> ["ts","UNIX\_MS"] target =\> "@timestamp" timezone =\> "America/New\_York" } } } GROK Patterns: CUSDATE [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3} TYPE [A-z\s]+ wso2carbon %{GREEDYDATA:tid}%{SPACE}[%{CUSDATE:timestamp}]%{SPACE}[%{GREEDYDATA:message}]%{SPACE}[%{GREEDYDATA:req\_id}]%{SPACE}%{WORD:loglevel}%{SPACE}{%{GREEDYDATA:class}}%{SPACE}-%{SPACE}%{TYPE:type}:%{GREEDYDATA:kvpairs} Log I am trying to filter: TID: [0] [AM] [2017-08-10 13:35:24,157] [PassThroughMessageProcessor-264] [FSREQID=sdasfdfsrgrwgfdfsafda] INFO {some\_url\_here} - Initiating Request : ClientIp = 10.xx.x.xx, ClientHost = myhost, xClientIp = [10.xxx.xx.xxx](http://10.xxx.xx.xxx), xForwardedFor = null, CorrelationID = xxxxxxxxddsdfcdsfewewdef, FSREQID = dsaefr24ffewefefre, HTTPMethod = GET, Url = some/url/here, ContentType = null I am testing my Grok filter using the GROK debugger and everything checks out. When I start up my LogStash it is not throwing any errors about the config. When i query the data in Kibana I do not see the \_grokparsefailure on the data. Not sure what is going on here any help would be appreciated. Thanks! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 10, 2017, 6:09pm UTC](https://discuss.elastic.co/t/logstash-config-not-parsing-out-log-events-even-though-config-checks-out-in-grok-debugger/96653/2 "2017-08-10T18:09:15Z")
Show what a raw event processed by Logstash looks like. Use a `stdout { codec => rubydebug }` output. ---
**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Post date:** [August 11, 2017, 9:23pm UTC](https://discuss.elastic.co/t/logstash-config-not-parsing-out-log-events-even-though-config-checks-out-in-grok-debugger/96653/3 "2017-08-11T21:23:24Z")
I was having issues getting it to print out on the terminal .... i had to change the regex that was parsing out the timestamp to {TIMESTAMP\_ISO8601:timestamp} and that actually parsed out to the console. I also changed the date stanza to: date { match =\> ["timestamp" ,"yyyy-MM-dd HH:mm:ss,SSS"] target =\> "@timestamp" timezone =\> "America/New\_York" } timestamp: "2017-08-11 17:10:22,212 Below you can see the output, but unfortunately it is still not showing up in Kibana parsed out. No grokparsefailure either. ``` "path" => "/apps/logstash-5.2.0/bin/test3.txt", "@timestamp" => 2017-08-11T21:20:31.586Z, "@version" => "1", "host" => "myhost", "log_level" => "WARN", "log_message" => "__SynapseService Executing fault sequence mediator : fault", "message" => "2017-08-11 17:10:22,212 [-] [PassThroughMessageProcessor-280] [] WARN __SynapseService Executing fault sequence mediator : fault", "messageB" => "PassThroughMessageProcessor-280", "timestamp" => "2017-08-11 17:10:22,212", "messageA" => "-" ``` Actual message: 2017-08-11 17:10:22,212 [-] [PassThroughMessageProcessor-280] [] WARN \_\_SynapseService Executing fault sequence mediator : fault ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [August 13, 2017, 6:36pm UTC](https://discuss.elastic.co/t/logstash-config-not-parsing-out-log-events-even-though-config-checks-out-in-grok-debugger/96653/4 "2017-08-13T18:36:15Z")
Where are the `service` and `[attributes][file_path]` fields? You only apply the grok filter if those fields have certain values. But _some_ grok filter or similar is obviously being used since you have fields like `log_level` and `log_message`. ---
**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Post date:** [August 14, 2017, 5:07pm UTC](https://discuss.elastic.co/t/logstash-config-not-parsing-out-log-events-even-though-config-checks-out-in-grok-debugger/96653/5 "2017-08-14T17:07:04Z")
For this particular test to the terminal I am not invoking the service and attributes field paths ... I am testing the GROK filter directly to the event I am trying to parse. The events is getting parsed out correctly. The service and attributes field path are specific to each of the events i want parsed out. The service will be the same for the four type of events i want parsed out, but the file path will be specific to the event. For example: else if ([service] == "my\_service"){ if([attributes][file\_path] == "/apps/log/my\_service/wso2-errors.log") { else if ([service] == "my\_service"){ if([attributes][file\_path] == "/apps/log/my\_service/wso2-service.log") { else if ([service] == "my\_service"){ if([attributes][file\_path] == "/apps/log/my\_service/gc.log") { else if ([service] == "my\_service"){ if([attributes][file\_path] == "/apps/log/my\_service/wso2carbon.log") { ---
**Author:** ![cchooks2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cchooks2/32/21987_2.png) [@cchooks2](https://discuss.elastic.co/u/cchooks2)\ **Post date:** [August 16, 2017, 8:51pm UTC](https://discuss.elastic.co/t/logstash-config-not-parsing-out-log-events-even-though-config-checks-out-in-grok-debugger/96653/6 "2017-08-16T20:51:08Z")
Figured this out. I was calling the same service in my else ifs. I just nested if's within the else and add the corresponding paths. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 13, 2017, 8:51pm UTC](https://discuss.elastic.co/t/logstash-config-not-parsing-out-log-events-even-though-config-checks-out-in-grok-debugger/96653/7 "2017-09-13T20:51:11Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.