# Logstash Grok Parsing Issue **URL:** \ **Category:** Logstash\ **Created:** [July 8, 2016, 3:08am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010 "2016-07-08T03:08:09Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Post date:** [July 8, 2016, 3:08am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/1 "2016-07-08T03:08:09Z")
I am using Logstash to read some log file. Here are some data sources records ``` <2016-07-07 00:31:01> Start <2016-07-07 00:31:59> Warning - Export_Sysem 6 (1) => No records to be exported <2016-07-07 00:32:22> Export2CICAP (04) => Export PO : 34 record(s) <2016-07-07 00:32:22> Export2CICAP (04) => Export CO : 87 record(s) <2016-07-07 00:32:22> Export2CICAP (04) => Export FC ``` This is my conf file ``` grok{ match => {"message" => [ '<%{TIMESTAMP_ISO8601:Timestamp}> (%{WORD:Level} - )%{NOTSPACE:Job_Code} => %{GREEDYDATA:Message}', '<%{TIMESTAMP_ISO8601:Timestamp}> %{WORD:Parameter} - %{GREEDYDATA:Message}', '<%{TIMESTAMP_ISO8601:Timestamp}> %{WORD:Status}', ]} } ``` This is part of my output ``` { "message" => "??2016-07-07 00:31:01> Start\r?", "@version" => "1", "@timestamp" => "2016-07-08T03:22:01.076Z", "path" => "C:/CIGNA/Export.log", "host" => "SIMSPad", "type" => "txt", "tags" => [ [0] "_grokparsefailure" ] } { "message" => "<2016-07-07 00:31:59> Warning - Export_Sysem 6 (1) => No records to be exported\r?", "@version" => "1", "@timestamp" => "2016-07-06T16:31:59.000Z", "path" => "C:/CIGNA/Export.log", "host" => "SIMSPad", "type" => "txt", "Timestamp" => "2016-07-07 00:31:59", "Parameter" => "Warning", "Message" => "Export_Sysem 6 (1) => No records to be exported\r?" } { "message" => "<2016-07-07 00:32:22> Export2CICAP (04) => Export CO : 87 record(s)\r?", "@version" => "1", "@timestamp" => "2016-07-06T16:32:22.000Z", "path" => "C:/CIGNA/Export.log", "host" => "SIMSPad", "type" => "txt", "Timestamp" => "2016-07-07 00:32:22", "Status" => "Export2CICAP" } ``` As seen from the output, the outcome did not fully parse the message. How should I modify my grok statement so it can fully parse the message? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 8, 2016, 5:30am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/2 "2016-07-08T05:30:16Z")
What does "fully parse the message" mean, exactly? Are there additional fields you want to extract? ---
**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Post date:** [July 8, 2016, 5:45am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/3 "2016-07-08T05:45:40Z")
As you seen from the first output, `<2016-07-07 00:31:01> Start` becomes `??2016-07-07 00:31:01> Start\r?` and it cannot be successfully parsed. Then as seen from the 3rd output`<2016-07-07 00:32:22> Export2CICAP (04) => Export CO : 87 record(s)\r?` has not fully parsed. Here is the expected output ``` { "message" => "<2016-07-07 00:32:22> Export2CICAP (04) => Export CO : 87 record(s)\r?", "@version" => "1", "@timestamp" => "2016-07-06T16:32:22.000Z", "path" => "C:/CIGNA/Export.log", "host" => "SIMSPad", "type" => "txt", "Timestamp" => "2016-07-07 00:32:22", "Job_Code" => "Export2CICAP (04)" "Message" => "Export CO : 87 record(s)" } ``` which is different from the current one. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 8, 2016, 5:55am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/4 "2016-07-08T05:55:56Z")
> As you seen from the first output, `<2016-07-07 00:31:01> Start` becomes `??2016-07-07 00:31:01> Start\r?` and it cannot be successfully parsed. Perhaps there's a byte-order mark or something at the beginning of the file. Check in a hex editor. > Then as seen from the 3rd output`<2016-07-07 00:32:22> Export2CICAP (04) => Export CO : 87 record(s)\r?` has not fully parsed. Here is the expected output Well, you don't have any grok expressions for matching that kind of message. Your first expression is close but still doesn't match. ---
**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Post date:** [July 8, 2016, 5:58am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/5 "2016-07-08T05:58:30Z")
> [@magnusbaeck](#): > > Well, you don't have any grok expressions for matching that kind of message. Your first expression is close but still doesn't match. Does that mean I can only graps the whole part of data, i.e.` Export2CICAP (04) => Export CO : 87 record(s)\r?` into a field? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 8, 2016, 6:00am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/6 "2016-07-08T06:00:00Z")
You can extract any parts of the data you like but you must write grok expressions for doing so. ---
**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Post date:** [July 8, 2016, 6:07am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/7 "2016-07-08T06:07:39Z")
Sorry, I don't quite get it. What does it mean by `you don't have any grok expressions for matching that kind of message.`? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [July 8, 2016, 6:21am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/8 "2016-07-08T06:21:50Z")
You have three grok expressions in your configuration. None of them work for the message you're asking about. The first expression almost matches but not quite. You need to adjust it or add another expression. I don't think I can explain this in any other way. Good luck. ---
**Author:** ![Kennedy\_Kan1](https://avatars.discourse-cdn.com/v4/letter/k/cc9497/32.png) [@Kennedy\_Kan1](https://discuss.elastic.co/u/Kennedy_Kan1)\ **Post date:** [July 8, 2016, 6:31am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/9 "2016-07-08T06:31:03Z")
Thanks for your replies. And as for [quote="magnusbaeck, post:4, topic:55010"] Perhaps there's a byte-order mark or something at the beginning of the file. Check in a hex editor. [/quote] How should I get rid of a BOM? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:48am UTC](https://discuss.elastic.co/t/logstash-grok-parsing-issue/55010/10 "2017-07-06T04:48:55Z")