Logstash uses the time in the log to resolve to @timestamp, the parsing format in the match is disassembled, and a matching exception occurs!

The format of the time field in the log:
version=1.0 time="2019-01-15 18:20:05" dev="WAF01.PUB.BEIJING-B" pri="0"

Parsing in match:

date {
  #"time" => "2019-01-15 18:20:05"
  match => ["time", "yyyy-MM-dd HH:mm:ss"]
  target => "@timestamp"
  locale => "en"
}

I corresponded in time format, but the reported exception information only shows the format as time, minute, and second.

error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [time]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2019-01-15 18:20:05\" is malformed at \" 18:20:05\""}

The normal format is considered to be:
" 18:20:05"

Does anyone know why this was dismantled?
I have seen the same problem encountered below, but did not make a reasonable explanation, my time is not Xia Li camp time.

This is really an elasticsearch question. The error is not logged by the date filter, it is logged by the elasticseach output. If you change the output to be stdout { codec => rubydebug } then I think you will see the timestamp was parsed OK.

I believe you could change the document mapping to accept that format, but as I said, it is an elasticsearch question.

This error message appears in the log of the logstash log, I use the stdout { codec => rubydebug } output. However, the corresponding index data is not generated in elasticsearch.

[WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"test", :_type=>"udp_test", :_routing=>nil}, #LogStash::Event:0x64ffbcdf], :response=>{"index"=>{"_index"=>"test", "_type"=>"udp_test", "_id"=>"1AhKVGgBcZL-34tzTvzI", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [time]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2019-01-15 18:20:05" is malformed at " 18:20:05""}}}}}

"time" => "2019-01-15 18:20:05"

but the timestamp is

"@timestamp" => 2019-01-16T01:38:17.021Z

The data line is parsed by the kv plugin, and then the date processing is performed.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.