We ingest a lot of custom logs in Elasticsearch. For the application logs we use a custom schema with dynamic mappings, but when sorting for some of the fields we hit a strange bug:
- Sort by a
dext.duration#double
field which is mapped todouble
by a dynamic mapping:
{
"double": {
"path_match": "dext.*#double",
"mapping": {
"type": "double"
}
}
},
- And we get a
Can't sort on field [__anonymous_]; the field has incompatible sort types: [DOUBLE] and [LONG] across shards!
. It reports as__anonymous_
.
- If I add a filter for
dext.duration#double exists
, see the screenshot, it returns results:
- I don't have documents where the field doesn't exist, so it's very strange behavior.
Can you advise what is the possible cause?
I can provide more debug information if needed, if you can guide me on how to debug it.