Logstash/elasticsearch failed to parse date field tried both date format [dateOptionalTime], and timestamp number with locale

Elastic Stack Logstash
#1

Hi,
for some reason, logstash (version 1.5) can't process logs with this exception:

{:timestamp=>"2016-04-26T09:20:12.141000-0400", :message=>"Failed parsing date from field", :field=>"time", :value=>"2016-04-26T09:20:03.520-04:00", :exception=>java.lang.IllegalArgumentException: Invalid format: "2016-04-26T09:20:03.520-04:00" is malformed at "T09:20:03.520-04:00", :level=>:warn}

My Time field in json is:

"time":"25-04-2016 04:21:06.786"

my logstash configuration is:

filter {

if [type] == "json" {
json {
source => "message"
}
date {
match => [ "time", "dd-MM-yyyy HH:mm:ss", "dd-MM-yyyy HH:mm:ss:SSS", "dd-MM-yyyy HH:mm:ss.SSS", "yyyy-MM-dd HH:mm:ss,SSS" ]
}
}
}

On Elasticsearch side I see this exception:

failed to parse date field [25-04-2016 04:48:14.305], tried both date format [dateOptionalTime], and timestamp number with locale [] java.lang.IllegalArgumentException: Invalid format: "25-04-2016 04:48:14.305" is malformed at "16 04:48:14.305"

How do I fix this? Isn't my "time" field suppose to replace "@timestamp" ?

Thanks!

#2

Judging by the first error message the time field contains "2016-04-26T09:20:12.141000-0400" and not "25-04-2016 04:21:06.786", and none of the date patterns you've supplied matches that. The last one is close, but doesn't expect the "T" separator between the date and the time (which the error message also indicates in the "... is malformed at ..." part).

#3

Thanks for the reply! So what does my pattern should look like? I can't just add 'T' to it, right?

#4

Almost. There's also the timezone. Something like yyyy-MM-dd'T'HH:mm:ss,SSSZZZ should be okay. Or, come to thing of it, the ISO8601 special case should also work.

#5

For some reason it doesn't work..
This is my configuration:

date {
match => [ "time", "yyyy-MM-dd'T'HH:mm:ss,SSSZZZ", "ISO8601" ]
}

Still getting these exceptions:

{:timestamp=>"2016-04-27T04:06:36.194000-0400", :message=>"Failed parsing date from field", :field=>"time", :value=>"25-04-2016 09:00:36.159", :exception=>java.lang.IllegalArgumentException: Invalid format: "25-04-2016 09:00:36.159" is malformed at "16 09:00:36.159", :level=>:warn}

#6

Your time apparently contains multiple date formats. With the current configuration you only support ISO8601 timestamp but you also need the ones you had previously.

#7

OK, I added back the previous formats as well:

date {
match => [ "time", "ISO8601", "dd-MM-yyyy HH:mm:ss", "dd-MM-yyyy HH:mm:ss:SSS", "dd-MM-yyyy HH:mm:ss.SSS", "yyyy-MM-dd HH:mm:ss,SSS" ]
}

Now I don't see errors in logstash regarding time issues, only - "failed action with response of 400, dropping action"

On ES side I still see malformed time issues:

failed to parse date field [25-04-2016 05:34:24.098], tried both date format [dateOptionalTime], and timestamp number with locale []

Invalid format: "25-04-2016 05:34:24.098" is malformed at "16 05:34:24.098"

So it looks like the problem moved from logstash to ES now?

Thanks for the help!

#8

Do you even need to keep the time field now that you've parsed it into @timestamp? If not, add remove_field to your date filter.

date {
  match => ["time", "..."]
  remove_field => ["time"]
}
#9

Looks like that solved it for me. Thanks for the help!

#10

Hello,

I am have this same issue and i have followed the solution provided here but not yet working.

Error:
"status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [@timestamp]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2016-09-08T13:06:42.003Z" is malformed at "T13:06:42.003Z""}}}}, :level=>:warn}.

logstash config:

date {
locale => "en"
match => [ "timestamp","ISO8601","yyyy-MM-dd'T'HH:mm:ss,SSSZZ" ]
remove_field => ["timestamp"]
}

grok {
  match => { "timestamp" => "%{YEAR:[@metadata][yyyy]}-%{MONTHNUM:[@metadata][MM]}-%{MONTHDAY:[@metadata][dd]} ([^\s]+)" }
}

I got passed this error by by updating elasticsearch mapping to the correct timestamp but after that logstash complaining of the following error:

{:timestamp=>"2016-09-08T14:40:46.982000+0100", :message=>"Failed parsing date from field", :field=>"timestamp", :value=>"2016-09-08 13:38:10.212151", :exception=>"Invalid format: "2016-09-08 13:38:10.212151" is malformed at " 13:38:10.212151"", :config_parsers=>"ISO8601,yyyy-MM-dd'T'HH:mm:ss.SSSZZ", :config_locale=>"default=en_US", :level=>:warn}

Someone pls help!

THnaks

#11