"reason"=>"failed to parse field [Time] of type [date] in document with id 'xxxxxxxxxxxxxxx'

Elastic Stack Logstash
1

I set config of logstash, But happen warning message.
Please, how to do resolve.

I want to send below log to Elasticsearch.
//////////////////
2021-03-17 10:27:50.115 +0900 26240 main : INFO com.tableausoftware.activemq.ActiveMQApp - OK
//////////////////

logstash config

grok {
 match => { "message" => "%{TIMESTAMP_ISO8601:Time} %{ISO8601_TIMEZONE} %{GREEDYDATA:Message}" }
}

warning message

[2021-03-17T19:27:55,351][WARN ][logstash.outputs.elasticsearch][main][9002db901028329940eb5fba931f1c415dff0a351278ecd055258afdbfcd922b] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"server_log", :routing=>nil, :_type=>"_doc"}, #<LogStash::Event:0x3606582c>], :response=>{"index"=>{"_index"=>"server_log", "_type"=>"_doc", "_id"=>"pqu6P3gBhQBy17rcRRFT", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [Time] of type [date] in document with id 'pqu6P3gBhQBy17rcRRFT'. Preview of field's value: '2021-03-17 10:27:00.115'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [2021-03-17 10:27:00.115] with format [strict_date_optional_time||epoch_millis]", "caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"Failed to parse with all enclosed parsers"}}}}}}
2

elasticsearch expects the [Time] field to be a date. Either you have a mapping that tells it that, or it has previously decided that using dynamic mapping.

The date parser defaults to strict_date_optional_time or epoch_millis.

strict_date_optional_time is "A generic ISO datetime parser, where the date must include the year at a minimum, and the time (separated by T), is optional. Examples: yyyy-MM-dd'T'HH:mm:ss.SSSZ or yyyy-MM-dd."

So you can use mutate+gsub to change the " " to "T" in "2021-03-17 10:27:00.115", or else use a date filter to parse the field and overwrite it, in which case the elasticsearch output will send it to elasticsearch in an appropriate format.

2 Likes
3

I added below config.
It work.

mutate+gsub

mutate {
   gsub => [ "Time", " ", "T" ]
 }

Thank you for your advise.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.