# Failed date from field **URL:** \ **Category:** Logstash\ **Created:** [March 2, 2016, 6:17pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294 "2016-03-02T18:17:14Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 2, 2016, 6:17pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/1 "2016-03-02T18:17:14Z")
I have the conf file reading data but still getting exceptions to my CSV Timestamp issues. Is this the proper ISO8601 format? 2016-02-21T04:15:04.290 Failed parsing date from field {:field=\>"mydatetime", :value=\>"2016-02-21T04:15:04.290", :exception=\>"Invalid format: "2016-02-21T04:15:04.290" is malformed at "T04:15:04.290"", :config\_parsers=\>"yyyy-MM-dd HH:mm:ss.SSS", :config\_locale=\>"en", :level=\>:warn} Other notes: > [@CSV Timstamp issues](https://discuss.elastic.co/t/csv-timstamp-issues/43050/15): > > This is what i have now. input { file { path =\> "/home/bkelley6/flights/\*.csv" type =\> "flights" start\_position =\> "beginning" } } filter { csv { columns =\> ["Date", "Time", "SWR", "RSSI(dB)", "RxBt(V)", "Cels(gRe)", "Tmp2(@C)", "RPM(rpm)", "Tmp1(@C)", "Rud", "Ele", "Thr", "Ail", "S1", "S2", "S3", "LS", "RS", "SA", "SB", "SC", "SD" ,"SE", "SF", "SG", "SH"] separator =\> "," } mutate { replace =\> ["date", "%{Date} %{Time}"] } date { "locale" =\> "en" match =\> [ "date", "YYYY-MM-… ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 2, 2016, 6:35pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/2 "2016-03-02T18:35:27Z")
So you're attempting to parse the string "2016-02-21T04:15:04.290" with the pattern "yyyy-MM-dd HH:mm:ss.SSS"? That won't work because your pattern doesn't consider the presence of the "T" between the date and the time. Use "yyyy-MM-dd'T'HH:mm:ss.SSS" instead. ---
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 2, 2016, 7:02pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/3 "2016-03-02T19:02:32Z")
Failed parsing date from field {:field=\>"mydatetime", :value=\>"2016-02-21T04:2016-02-21", :exception=\>"Invalid format: "2016-02-21T04:2016-02-21" is malformed at "16-02-21"", :config\_parsers=\>"yyyy-MM-dd'T'HH:mm:ss.SSS", :config\_locale=\>"en", :level=\>:warn} Here is what I currently have: input { file { path =\> "/home/bkelley6/flights/\*.csv" type =\> "flights" start\_position =\> "beginning" } } filter { csv { columns =\> ["Date", "Time", "SWR", "RSSI(dB)", "RxBt(V)", "Cels(gRe)", "Tmp2(@C)", "RPM(rpm)", "Tmp1(@C)", "Rud", "Ele", "Thr", "Ail", "S1", "S2", "S3", "LS", "RS", "SA", "SB", "SC", "SD" ,"SE", "SF", "SG", "SH"] separator =\> "," } mutate { replace =\> ["mydatetime", "%{Date}T%{Time}"] } date { locale =\> "en" match =\> ["mydatetime", "yyyy-MM-dd'T'HH:mm:ss.SSS"] timezone =\> "America/New\_York" target =\> ["@timestamp"] } } output { elasticsearch { action =\> "index" hosts =\> ["localhost:9200"] index =\> "logstash-%{+YYYY.MM.dd}" workers =\> 1 } } ---
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 2, 2016, 7:03pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/4 "2016-03-02T19:03:25Z")
sample of the CSV file: Date,Time,SWR,RSSI(dB),RxBt(V),Cels(gRe),Tmp2(@C),RPM(rpm),Tmp1(@C),Rud,Ele,Thr,Ail,S1,S2,S3,LS,RS,SA,SB,SC,SD,SE,SF,SG,SH, 2016-02-21,04:11:14.640,30,75,5.2,23.4,0,0,0,0,0,0,0,0,0,0,0,0,-1,-1,-1,-1,-1,-1,-1,-1, 2016-02-21,04:11:14.840,30,75,5.2,23.4,0,0,0,0,0,0,0,0,0,0,0,0,-1,-1,-1,-1,-1,-1,-1,-1, ---
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 2, 2016, 7:04pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/5 "2016-03-02T19:04:11Z")
Thanks for all your help. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 2, 2016, 7:07pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/6 "2016-03-02T19:07:53Z")
Works fine for me. ```auto $ cat test.config filter { csv { columns => ["Date", "Time", "SWR", "RSSI(dB)", "RxBt(V)", "Cels(gRe)", "Tmp2(@C)", "RPM(rpm)", "Tmp1(@C)", "Rud", "Ele", "Thr", "Ail", "S1", "S2", "S3", "LS", "RS", "SA", "SB", "SC", "SD" ,"SE", "SF", "SG", "SH"] separator => "," } mutate { replace => ["mydatetime", "%{Date}T%{Time}"] } date { locale => "en" match => ["mydatetime", "yyyy-MM-dd'T'HH:mm:ss.SSS"] timezone => "America/New_York" target => ["@timestamp"] } } input { stdin {} } output { stdout { codec => rubydebug } } $ echo '2016-02-21,04:11:14.640,30,75,5.2,23.4,0,0,0,0,0,0,0,0,0,0,0,0,-1,-1,-1,-1,-1,-1,-1,-1,' | /opt/logstash/bin/logstash -f test.config Settings: Default pipeline workers: 2 Logstash startup completed { "message" => "2016-02-21,04:11:14.640,30,75,5.2,23.4,0,0,0,0,0,0,0,0,0,0,0,0,-1,-1,-1,-1,-1,-1,-1,-1,", "@version" => "1", "@timestamp" => "2016-02-21T09:11:14.640Z", "host" => "hallonet", "Date" => "2016-02-21", "Time" => "04:11:14.640", "SWR" => "30", "RSSI(dB)" => "75", "RxBt(V)" => "5.2", "Cels(gRe)" => "23.4", "Tmp2(@C)" => "0", "RPM(rpm)" => "0", "Tmp1(@C)" => "0", "Rud" => "0", "Ele" => "0", "Thr" => "0", "Ail" => "0", "S1" => "0", "S2" => "0", "S3" => "0", "LS" => "0", "RS" => "0", "SA" => "-1", "SB" => "-1", "SC" => "-1", "SD" => "-1", "SE" => "-1", "SF" => "-1", "SG" => "-1", "SH" => "-1", "column27" => nil, "mydatetime" => "2016-02-21T04:11:14.640" } Logstash shutdown completed ``` ---
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 2, 2016, 7:32pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/7 "2016-03-02T19:32:28Z")
Would it cause an issue if I am taking in the first line? Brad ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 2, 2016, 7:38pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/8 "2016-03-02T19:38:23Z")
Yes, but not _that_ kind of error. ---
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 2, 2016, 7:43pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/9 "2016-03-02T19:43:50Z")
when I add the output to the conf: output { elasticsearch { action =\> "index" hosts =\> ["localhost:9200"] index =\> "logstash-%{+YYYY.MM.dd}" workers =\> 1 } } I get the error messages? Failed parsing date from field {:field=\>"mydatetime", :value=\>"2016-02-21T04:2016-02-21", :exception=\>"Invalid format: "2016-02-21T04:2016-02-21" is malformed at "16-02-21"", :config\_parsers=\>"yyyy-MM-dd'T'HH:mm:ss.SSS", :config\_locale=\>"en", :level=\>:warn} The debug worked great for me too!! Not sure that is the issue? Thanks for your help, Brad ---
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 2, 2016, 8:09pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/10 "2016-03-02T20:09:16Z")
Do I need to remove the mydatetime variable? Thanks, Brad ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [March 2, 2016, 8:34pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/11 "2016-03-02T20:34:14Z")
I'm not sure what's going on here, but given that you parse the timestamp into `@timestamp` I don't see why you'd want to keep `mydatetime`. ---
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 2, 2016, 8:57pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/12 "2016-03-02T20:57:49Z")
Failed parsing date from field {:field=\>"mydatetime", :value=\>"2016-02-21T17:2016-02-21", :exception=\>"Invalid format: "2016-02-21T17:2016-02-21" is malformed at "16-02-21"", :config\_parsers=\>"yyyy-MM-dd'T'HH:mm:ss.SSS", :config\_locale=\>"en", :level=\>:warn} this error is only when i output to elasticsearch? output { elasticsearch { action =\> "index" hosts =\> ["localhost:9200"] index =\> "logstash-%{+YYYY.MM.dd}" workers =\> 1 } } ---
**Author:** ![hursto75](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hursto75/32/8168_2.png) [@hursto75](https://discuss.elastic.co/u/hursto75)\ **Post date:** [March 3, 2016, 2:49pm UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/13 "2016-03-03T14:49:23Z")
Little more information and some big hints to the problem. I wrote a little different config file (notice the input method): filter { csv { columns =\> ["Date", "Time", "SWR", "RSSI(dB)", "RxBt(V)", "Cels(gRe)", "Tmp2(@C)", "RPM(rpm)", "Tmp1(@C)", "Rud", "Ele", "Thr", "Ail", "S1", "S2", "S3", "LS", "RS", "SA", "SB", "SC", "SD" ,"SE", "SF", "SG", "SH"] separator =\> "," } mutate { replace =\> ["mydatetime", "%{Date}T%{Time}"] } date { locale =\> "en" match =\> ["mydatetime", "yyyy-MM-dd'T'HH:mm:ss.SSS"] timezone =\> "America/New\_York" target =\> ["@timestamp"] remove\_field =\> ["mydatetime"] } } input { stdin {} } output { elasticsearch { action =\> "index" hosts =\> ["localhost:9200"] index =\> "logstash-%{+YYYY.MM.dd}" workers =\> 1 } } And wrote this one liner to process one line at a time. cat /home/bkelley6/flights/T-Rex\_500-2016-02-21-1-test.csv | while read line ; do echo $line | /opt/logstash/bin/logstash -f magnus-elastic.conf; done This works! So I am thinking the problem is with the input on the the other conf file: input { file { path =\> "/home/bkelley6/flights/\*.csv" type =\> "flights" start\_position =\> "beginning" } } Why would this be causing a problem? Once again thank you for all your help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:08am UTC](https://discuss.elastic.co/t/failed-date-from-field/43294/14 "2017-07-06T05:08:29Z")