Failed to parse date field, date_time_parse_exception

Elastic Stack Elasticsearch
1

Here is my configuration:
"[log][timestamp]" => "%{[yearinlog]}-%{[monthinlog]}-%{[dateinlog]} %{[hourinlog]}:%{[minuteinlog]}:%{[secondinlog]}"
}
}
date {
match => [
"[log][timestamp]",
"yyyy-MM-dd HH:mm:ss.SSS000"
]
timezone => "Asia/Singapore"
target => "@timestamp"
}

I am trying to parse into elastic search but i am getting these as shown below. Any ideas how I can fix this?

Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-mneta-app-2021.01.14", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x3091220c], :response=>{"index"=>{"_index"=>"filebeat-mneta-app-2021.01.14", "_type"=>"_doc", "_id"=>"dEfHGHcBobdQVkyHZgd2", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [log.timestamp] of type [date] in document with id 'dEfHGHcBobdQVkyHZgd2'. Preview of field's value: '2021-01-15 03:32:11.874'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [2021-01-15 03:32:11.874] with format [strict_date_optional_time||epoch_millis]", "caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"Failed to parse with all enclosed parsers"}}}}}}

2

You can see the format of strict_date_optional_time here - https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html

What is the mapping of the log.timestamp field?

3

2014-05-12 17:10:06.209000

here? im not sure what can i do to resolve this

4

That is not the mapping though.

5

what is considered as mapping though? sorry

6

GET filebeat-mneta-app-2021.01.14/_mapping in tghe Kibana console will show it.

7

oh thanks. here is my mapping

{"filebeat-mneta-app-2021.01.14":{"mappings":{"properties":{"@timestamp":{"type":"date"},"@version":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"agent":{"properties":{"ephemeral_id":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"hostname":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"id":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"type":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"version":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"dateinpath":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"ecs":{"properties":{"version":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"event":{"properties":{"app":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"created":{"type":"date"},"dataset":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"host":{"properties":{"name":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"impact":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"input":{"properties":{"type":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"log":{"properties":{"file":{"properties":{"path":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"offset":{"type":"long"},"timestamp":{"type":"date"}}},"log_level":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"log_message":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"monthinpath":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"scenario":{"type":"keyword","fields":{"keyword":{"type":"keyword"}}},"tags":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"yearinpath":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}}}}

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.