# Date parse failure and date formatter issue **URL:** \ **Category:** Logstash\ **Created:** [November 8, 2017, 7:46pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925 "2017-11-08T19:46:18Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![guruprasadh4](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Post date:** [November 8, 2017, 7:46pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/1 "2017-11-08T19:46:19Z")
Sample log - IDENTIFIER TIMESTAMP T C RESOURCE\_NAME DESCRIPTION C6E26F3B 1002042417 I H hdisk111 BACK-UP PATH STATUS CHANGE C6E26F3B 1002042417 I H hdisk103 BACK-UP PATH STATUS CHANGE Config File - input { file { path =\> "/home/elkuser/KP/LogData/errpt/server\_lun\_info1.log" #path =\> "/home/elkuser/KP/LogData/errpt/errpt1.log" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { grok { # match =\> ["message", "\A%{HOSTNAME:IDENTIFIER}%{SPACE}+%{NUMBER:TIMESTAMP}%{SPACE}+%{PROG:T}%{SPACE}+%{PROG:C}%{SPACE}+%{PROG:TIMESTAMP}%{SPACE}+%{GREEDYDATA:DESCRIPTION}"] match =\> ["message", "\A%{PROG:IDENTIFIER}%{SPACE}+%{NUMBER:TIMESTAMP}%{SPACE}+%{NOTSPACE:T}%{SPACE}+%{NOTSPACE:C}%{SPACE}+%{NOTSPACE:RESOURCE\_NAME}%{SPACE}+%{GREEDYDATA:DESCRIPTION}"] overwrite =\> ["message"] } mutate { convert =\> {"TIMESTAMP" =\> "integer"} } date{ match =\> ["TIMESTAMP", "MMddHHmmyy"] target =\> "TIMESTAMP" } #date\_formatter { # source =\> "TIMESTAMP" # target =\> "timestamp1" # pattern =\> "MM/DD/YY HH:mm" # # locale =\> "fr-FR" # timezone =\> "PST" #} } output { elasticsearch { hosts =\> "localhost" index =\> "kp-epims-errpt" # index =\> "kp-epims-errpt1" ``` template_overwrite => true } stdout { codec => rubydebug } } ``` **Error Message 🙂** "path" =\> "/home/elkuser/KP/LogData/errpt/server\_lun\_info1.log", "@timestamp" =\> 2017-11-08T19:38:47.818Z, "C" =\> "S", "T" =\> "T", "DESCRIPTION" =\> "SOFTWARE PROGRAM ERROR", "TIMESTAMP" =\> 1001234217, "@version" =\> "1", "host" =\> "[analyticslab72.in.ibm.com](http://analyticslab72.in.ibm.com)", "IDENTIFIER" =\> "DC73C03A", "message" =\> "DC73C03A 1001234217 T S fscsi0 SOFTWARE PROGRAM ERROR", "RESOURCE\_NAME" =\> "fscsi0", "tags" =\> [ [0] "\_dateparsefailure" ] 1. trying to match MMddHHmmyy and getting dateparse failure. 2. from MMddHHmmyy trying to convert to another format MM/dd/yy HH:mm using date\_formatter and not able to do so. 3. how to convert date from one format to the required format. Please advise. Thanks! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 9, 2017, 7:06am UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/2 "2017-11-09T07:06:42Z")
When the date filter fails it'll log a message and point to where the problem is. ---
**Author:** ![guruprasadh4](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Post date:** [November 9, 2017, 1:00pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/3 "2017-11-09T13:00:16Z")
Not sure, can you tell in the above message where it points to erro ? Thanks! ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [November 9, 2017, 1:22pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/4 "2017-11-09T13:22:55Z")
I'm not sure what you mean. Just check your Logstash log for entries from the date filter. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 9, 2017, 3:38pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/5 "2017-11-09T15:38:39Z")
> [@guruprasadh4](#): > > mutate { > convert =\> {"TIMESTAMP" =\> "integer"} > } Delete this. date wants to parse a string. @magnusbaeck, I do not see any message in the logstash logs when I test this. ---
**Author:** ![guruprasadh4](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Post date:** [November 9, 2017, 6:06pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/6 "2017-11-09T18:06:34Z")
I did removed the mutate, still dateparse error. ], "@version" =\> "1", "host" =\> "[oc1008401175.ibm.com](http://oc1008401175.ibm.com)", "IDENTIFIER" =\> "DC73C03A", "message" =\> "DC73C03A 1001234217 T S fscsi2 SOFTWARE PROGRAM ERROR", "tags" =\> [ [0] "\_dateparsefailure" ] } { "path" =\> "/home/gprasad9/KP/LogData/errpt/server\_lun\_info1.log", "@timestamp" =\> 2017-11-09T16:54:02.667Z, "C" =\> "S", "T" =\> "T", "DESCRIPTION" =\> "SOFTWARE PROGRAM ERROR", "TIMESTAMP" =\> [ [0] "1001234217", [1] "fscsi2" ], "@version" =\> "1", "host" =\> "[oc1008401175.ibm.com](http://oc1008401175.ibm.com)", "IDENTIFIER" =\> "DC73C03A", "message" =\> "DC73C03A 1001234217 T S fscsi2 SOFTWARE PROGRAM ERROR", "tags" =\> [ [0] "\_dateparsefailure" ] } { "path" =\> "/home/gprasad9/KP/LogData/errpt/server\_lun\_info1.log", "@timestamp" =\> 2017-11-09T16:54:02.668Z, "C" =\> "S", "T" =\> "T", "DESCRIPTION" =\> "SOFTWARE PROGRAM ERROR", "TIMESTAMP" =\> [ [0] "1001234217", [1] "fscsi0" ], "@version" =\> "1", "host" =\> "[oc1008401175.ibm.com](http://oc1008401175.ibm.com)", "IDENTIFIER" =\> "DC73C03A", "message" =\> "DC73C03A 1001234217 T S fscsi0 SOFTWARE PROGRAM ERROR", "tags" =\> [ [0] "\_dateparsefailure" Modified Config file as below : input { file { path =\> "/home/elkuser/KP/LogData/errpt/server\_lun\_info1.log" #path =\> "/home/elkuser/KP/LogData/errpt/errpt1.log" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" } } filter { grok { # match =\> ["message", "\A%{HOSTNAME:IDENTIFIER}%{SPACE}+%{NUMBER:TIMESTAMP}%{SPACE}+%{PROG:T}%{SPACE}+%{PROG:C}%{SPACE}+%{PROG:TIMESTAMP}%{SPACE}+%{GREEDYDATA:DESCRIPTION}"] match =\> ["message", "\A%{PROG:IDENTIFIER}%{SPACE}+%{NUMBER:TIMESTAMP}%{SPACE}+%{NOTSPACE:T}%{SPACE}+%{NOTSPACE:C}%{SPACE}+%{NOTSPACE:RESOURCE\_NAME}%{SPACE}+%{GREEDYDATA:DESCRIPTION}"] overwrite =\> ["message"] } mutate { convert =\> {"TIMESTAMP" =\> "integer"} } #ruby { code =\> "event.set('TIMESTAMP', event.get('TIMESTAMP') / 1)" } date{ match =\> ["TIMESTAMP", "MMddHHmmyy"] target =\> "TIMESTAMP" } } output { elasticsearch { hosts =\> "9.109.184.72:9200" index =\> "kp-epims-errpt" template\_overwrite =\> true } stdout { codec =\> rubydebug } } Sample log data DC73C03A 1001234217 T S fscsi0 SOFTWARE PROGRAM ERROR DC73C03A 1001234217 T S fscsi0 SOFTWARE PROGRAM ERROR DC73C03A 1001234217 T S fscsi0 SOFTWARE PROGRAM ERROR DC73C03A 1001234217 T S fscsi0 SOFTWARE PROGRAM ERROR DC73C03A 1001234217 T S fscsi2 SOFTWARE PROGRAM ERROR DC73C03A 1001234217 T S fscsi2 SOFTWARE PROGRAM ERROR DC73C03A 1001234217 T S fscsi2 SOFTWARE PROGRAM ERROR DC73C03A 1001234217 T S fscsi2 SOFTWARE PROGRAM ERROR DC73C03A 1001234117 T S fscsi2 SOFTWARE PROGRAM ERROR Also I was trying to change the data format : from MMddHHmmyy to MM/dd/yy HH:mm still no able to change the format. Any help would be greatly appreciated. Thanks! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [November 9, 2017, 6:33pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/7 "2017-11-09T18:33:35Z")
> [@guruprasadh4](#): > > "TIMESTAMP" =\> [ > > [0] "1001234217", > > [1] "fscsi0" > > ] This appears to be output from a run where TIMESTAMP was an array of strings. Again, not what the date filter is looking for. And the config that you are showing still contains that mutate the converts TIMESTAMP to an integer. Can you show input, output, and config that match please. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 9, 2017, 6:43pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/8 "2017-11-09T18:43:54Z")
> [@guruprasadh4](#): > > mutate { > convert =\> {"TIMESTAMP" =\> "integer"} > } The date filter matches a string, so why are you converting this to an integer? ---
**Author:** ![guruprasadh4](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Post date:** [November 9, 2017, 8:31pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/9 "2017-11-09T20:31:53Z")
Sorry. here is the updated config file and messages. input { file { path =\> "/home/gprasad9/KP/LogData/errpt/server\_lun\_info2.log" start\_position =\> "beginning" sincedb\_path =\> "/dev/null" ``` } ``` } filter { grok { match =\> ["message", "\A%{HOSTNAME:IDENTIFIER}%{SPACE}%{NUMBER:TIMESTAMP}%{SPACE}%{PROG:T}%{SPACE}%{PROG:C}%{SPACE}%{PROG:TIMESTAMP}%{SPACE}%{GREEDYDATA:DESCRIPTION}"] overwrite =\> ["message"] } date{ match =\> ["TIMESTAMP", "MMddHHmmyy"] target =\> "TIMESTAMP" } } output { elasticsearch { hosts =\> "localhost" index =\> "kp-epims-errpt" template\_overwrite =\> true } stdout {codec =\> rubydebug} } **Dateparse failure message:** ``` "path" => "/home/gprasad9/KP/LogData/errpt/server_lun_info2.log", "@timestamp" => 2017-11-09T20:26:53.669Z, "C" => "S", "T" => "T", "DESCRIPTION" => "ERROR LOGGING BUFFER OVERFLOW", "TIMESTAMP" => [ [0] "1001234517", [1] "syserrlg" ], "@version" => "1", "host" => "oc1008401175.ibm.com", "IDENTIFIER" => "A39F8A49", "message" => "A39F8A49 1001234517 T S syserrlg ERROR LOGGING BUFFER OVERFLOW", "tags" => [ [0] "_dateparsefailure" ] ``` ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 9, 2017, 8:45pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/10 "2017-11-09T20:45:39Z")
> [@guruprasadh4](#): > > %{PROG:TIMESTAMP} You have the `TIMESTAMP` field specified twice in the grok expression, which I suspect its why you get an array. ---
**Author:** ![guruprasadh4](https://avatars.discourse-cdn.com/v4/letter/g/cab0a1/32.png) [@guruprasadh4](https://discuss.elastic.co/u/guruprasadh4)\ **Post date:** [November 9, 2017, 10:01pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/11 "2017-11-09T22:01:02Z")
Oops!! fixed that and its fine Thanks a lot for your help! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 7, 2017, 10:01pm UTC](https://discuss.elastic.co/t/date-parse-failure-and-date-formatter-issue/106925/12 "2017-12-07T22:01:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.