I recently changed the logging configuration in my .NET application to use the official Serilog ECS formatter from Elastic (based on the recommendation in this official documentation).
Now when I go to the discover app in Kibana, and select the "logs-*" indices (which at this moment only matches indices created from this application), I see no data, and this error message pops up:
1 of 2 shards failed
The data you are seeing might be incomplete or wrong.
When I click show details, I see that the failure is due to an illegal_argument_exception, with this message:
Field [event.created] of type [keyword] does not support custom formats
Looking at the log events coming in using the logs app in Kibana (found under APM), I see logs with the event.created field, with a value that looks like this: 2020-09-30T16:24:02.4797966+00:00
This is a brand new Elastic Cloud observability stack. We are evaluating the product for our observability needs.
Do I need to configure something under index management if I want to use ECS? I already deleted all the indices, think a new index created from these log events will solve the problem. But it did not help.
Edit:
I noticed that this field (event.created) is mapped as "type": "keyword", and that the error in Kibana shows a 'request' (not sure what the request is, but whatever it is the discover app does) with:
"docvalue_fields":[
{
"field": "event.created",
"format": "date_time"
},
]
So I see a discrepancy between "type": "keyword" in the mapping and "format": "date_time" in this request.
Trying to find a solution to my problem I've found mentions in the official documentation of "ECS index template" but no specific topic. Do I need to create an index template to be able to index documents that use ECS? If so, where can I obtain this index template?