I encountered what appears to be a "gotcha" with dates. Wondering if this is expected behavior, or potentially room for improvement.
If you index a float value into a date field, by default Elasticsearch allows it. And, in fact, searches and aggregations work as expected. In my case, it appears that it chopped the fractional part, and accepted the rest as an epoch_millis date, which fit my need.
The problem comes later when you want to, for example, update the document with a new field. Or reindex the entire index. Elasticsearch errors with 'mapper_parsing_exception'.
These three steps reproduce the problem for me (in the Kibana Dev Tools console):
PUT /test/
{
"mappings": {
"properties": {
"some_date": {
"type": "date"
}
}
}
}
POST /test/_create/1
{
"some_date": 1616619193352.3
}
POST /test/_update/1
{
"doc": {
"some_text": "just a test"
}
}
Which returns:
{
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "failed to parse field [some_date] of type [date] in document with id '1'. Preview of field's value: '1.6166191933523E12'"
}
],
"type" : "mapper_parsing_exception",
"reason" : "failed to parse field [some_date] of type [date] in document with id '1'. Preview of field's value: '1.6166191933523E12'",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "failed to parse date field [1.6166191933523E12] with format [strict_date_optional_time||epoch_millis]",
"caused_by" : {
"type" : "date_time_parse_exception",
"reason" : "date_time_parse_exception: Failed to parse with all enclosed parsers"
}
}
},
"status" : 400
}
Of course, the floats that were indexed were a mistake. That has been fixed now. It just seems to me that the treatment of float values ought to be consistent. Either error out during document indexing, to indicate the value is not a valid date. Or, during update/re-index, allow the same kind of flexible treatment of date values. Allowing floats, but truncating the fraction as during initial indexing.
If I had my choice, I would have preferred that Elasticsearch reject these floats upfront during indexing, so that I would have known about the error sooner.
Curious to know if anyone else has encountered this, or if perhaps I'm missing something. Is there a way to make Elasticsearch more restrictive on the types of values it allows as dates during indexing?