Logstash uses the time in the log to resolve to @timestamp, the parsing format in the match is disassembled, and a matching exception occurs!

Elastic Stack Logstash
1

The format of the time field in the log:
version=1.0 time="2019-01-15 18:20:05" dev="WAF01.PUB.BEIJING-B" pri="0"

Parsing in match:

date {
  #"time" => "2019-01-15 18:20:05"
  match => ["time", "yyyy-MM-dd HH:mm:ss"]
  target => "@timestamp"
  locale => "en"
}

I corresponded in time format, but the reported exception information only shows the format as time, minute, and second.

error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [time]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2019-01-15 18:20:05\" is malformed at \" 18:20:05\""}

The normal format is considered to be:
" 18:20:05"

Does anyone know why this was dismantled?
I have seen the same problem encountered below, but did not make a reasonable explanation, my time is not Xia Li camp time.

2

This is really an elasticsearch question. The error is not logged by the date filter, it is logged by the elasticseach output. If you change the output to be stdout { codec => rubydebug } then I think you will see the timestamp was parsed OK.

I believe you could change the document mapping to accept that format, but as I said, it is an elasticsearch question.

3

This error message appears in the log of the logstash log, I use the stdout { codec => rubydebug } output. However, the corresponding index data is not generated in elasticsearch.

[WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"test", :_type=>"udp_test", :_routing=>nil}, #LogStash::Event:0x64ffbcdf], :response=>{"index"=>{"_index"=>"test", "_type"=>"udp_test", "_id"=>"1AhKVGgBcZL-34tzTvzI", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [time]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2019-01-15 18:20:05" is malformed at " 18:20:05""}}}}}

"time" => "2019-01-15 18:20:05"

but the timestamp is

"@timestamp" => 2019-01-16T01:38:17.021Z

The data line is parsed by the kv plugin, and then the date processing is performed.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.