Hello, I am trying to understand the following error:
The main problem is at the bottom of the error:
Invalid format: \"2018-03-11 02:48:31\" is malformed at \" 02:48:31\"
[2018-03-21T15:59:35,601][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>"1995397226128", :_index=>"unifieddxcvpc", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x3863e85>], :response=>{"index"=>{"_index"=>"unifieddxcvpc", "_type"=>"doc", "_id"=>"1995397226128", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [datetime]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2018-03-11 02:48:31\" is malformed at \" 02:48:31\""}}}}}
The logstash date mapping does handle this date format. Here is my mapping:
date {
target => [ "datetime" ]
match => [ "datetime", "yyyy-MM-dd", "yyyy-MM-dd HH:mm:ss", "yyyy-MM-dd:HH:mm:ss" ]
}
So I don't see why logstash will consider this data malformed. Here is the entry the error refers to (you can see "2018-03-11 02:48:31" clearly in here:
"EMEA";"Spain";"Barcelona";"ACR";"NONE";"VDI";"NONE";"NONE";"NONE";"domain";"Compute-DC";"text1";"1995397225952";"2018-03-11 02:48:31";"_TX4O0CTOEeiiGuzjU82SIA";"SP102-HAVS-LG-00203";"4232c628-94a1-032c-7665-476329da6cac";"MOVE";"PENDING_ACCEPT";"MANUAL";"30";"0";"0";"l2a2-3b19-u03-b08.acr.ecs.hp.com";"l2a2-3b19-u14-b08.acr.ecs.hp.com";"Mem congestion on Physical Machine 'l2a2-3b19-u03-b08.acr.ecs.hp.com'";"False";"Mem Congestion";"Performance Assurance";"Major"
I will try to change the order of the mapping matches, but it's supposed to keep trying all the mappings until it succeeds, no?
Your advice is much appreciated.
Best,
JD