Failed to parse field [eAgent.tstEvent] of type [date]

[vaclav1]

Dear team,

I have used date filter plugin to parse the date field:

date {
  	match => [ "[message][tstEventPlan]", "yyyy-MM-dd HH:mm:ss'.'SSS" ]
  	remove_field => [ "[message][tstEventPlan]" ]
  	target => "[eAgent][tstEventPlan]"
    tag_on_failure => "_dateparsefailure"
}

But got an error:

[2021-06-10T15:16:32,569][WARN ][logstash.outputs.elasticsearch][main][6771b6c1505e1d68ae197acd17a5ef39974052336877cc4d69a93d23ef9469dd] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"gelogext-lux", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x68f7b966>], :response=>{"index"=>{"_index"=>"gelogext-lux-2021.06.10-000001", "_type"=>"_doc", "_id"=>"aIgR9nkBjZSEu97_MdqM", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [eAgent.tstEvent] of type [date] in document with id 'aIgR9nkBjZSEu97_MdqM'. Preview of field's value: '2021-06-10 13:16:29.550'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [2021-06-10 13:16:29.550] with format [date_time]", "caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"date_time_parse_exception: Failed to parse with all enclosed parsers"}}}}}}

Example of the value is: 2021-06-10 13:16:29.550

[Badger]

"caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [2021-06-10 13:16:29.550] with format [date_time]"

Your elasticsearch index has a template that tells it that [eAgent][tstEvent] (not [eAgent][tstEventPlan]) should be a date_time. The format documentation says that date_time is

A formatter that combines a full date and time, separated by a T : yyyy-MM-dd'T'HH:mm:ss.SSSZZ .

and "2021-06-10 13:16:29.550" does not match that pattern so it cannot be parsed. Change your template, or mutate+gsub your field into compliance with the requirements of the parser.