Failed to parse date field error

[ssi]

hi i get these errors with one of the indexes (microsoft defender ATP data)

[2019-08-08T15:57:01,663][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-mdatp-2019.08.08", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x72f927bb], :response=>{"index"=>{"_index"=>"logstash-mdatp-2019.08.08", "_type"=>"_doc", "_id"=>"Zs-EcWwBes_2bu02wssO", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [records.properties.InitiatingProcessParentCreationTime] of type [date] in document with id 'Zs-EcWwBes_2bu02wssO'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [08/07/2019 12:32:32] with format [strict_date_optional_time||epoch_millis]", "caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"Failed to parse with all enclosed parsers"}}}}}}

i looked round the forum and though i had found a solution here

The curse of dynamic dates

No, I was not suggesting to change the "Z" in your input data. I was suggesting to replace the "Z" with an "X" in the date formats in your index template. The following works for me: PUT _template/tpl { "template": "myindex.*", "mappings": { "dynamic_date_formats": [ "basic_ordinal_date_time", "yyyy-MM-dd'T'HH:mm:ss.SX", "yyyy-MM-dd'T'HH:mm:ss.SSX", "yyyy-MM-dd'T'HH:mm:ss.SSSX" ] } } PUT myindex.1/_doc/1 { "my_date": "2018-12-23T04:48:08.123Z" } GET my…

PUT _template/tpl
{
"index_patterns": "myidx.*",
"mappings": {
"dynamic_date_formats": [
"yyyy-MM-dd'T'HH:mm:ssX||yyyy-MM-dd'T'HH:mm:ss.SX||yyyy-MM-dd'T'HH:mm:ss.SSX||yyyy-MM-dd'T'HH:mm:ss.SSSX"
]
}
}
but i still get the error do i have do fix something in the logstash config with grok / mutate or what?

the data is comming in fine i just get a ton of warning msg, so i want to get rid of those.

[Badger]

@ssi

08/07/2019 12:32:32

I think adding a template is the right approach, but none of the formats you have in your template match that. Try adding 'MM/dd/yyyy HH:mm:ss' in dynamic_date_formats.