Hi,
I've migrated my clusters from ES 6.4 to ES 6.7.1.
On these I have some indices which have some geo_shape fields. The mappings were defined using the "old" geo_shape parameters:
{
"location": {
"type": "geo_shape",
"tree": "quadtree",
"precision": "50.0m"
}
}
I migrated all those fields to the "new" geo_shape field format by simply changing them to:
{
"location": {
"type": "geo_shape",
}
}
All the impacted indices were then re-indexed to the new mappings using the reindex API.
Once the reindex was done I noticed that some documents couldn't be reindexed to the new field.
This can be reproduced on a 6.7 cluster (see this gist for the exact geojson shape):
PUT places-old
{
"mappings": {
"_doc": {
"dynamic": "strict",
"properties": {
"location": {
"type": "geo_shape",
"tree": "quadtree",
"precision": "50.0m"
}
}
}
}
}
POST places-old/_doc
{
"location": {
"type": "MultiPolygon",
"coordinates": [
[
[
[
//... see linked shape ...//
]
]
]
}
}
This works without issues. By the way it seems that the standard "upper-case" geometry notation type can now be used, which wasn't the case in ES 6.4 and I can't seem to find something related to this change in the documentation.
{
"_index" : "places-old",
"_type" : "_doc",
"_id" : "TU6zMGoBeEI9SuPC7thc",
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
Now trying to index this doc using the new geo_shape field type fails.
PUT places-new
{
"mappings": {
"_doc": {
"dynamic": "strict",
"properties": {
"location": {
"type": "geo_shape"
}
}
}
}
}
POST places-new/_doc
{
"location": {
"type": "MultiPolygon",
"coordinates": [
[
[
[
//... see linked shape ...//
]
]
]
}
}
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failed to parse field [location] of type [geo_shape]"
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse field [location] of type [geo_shape]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Unable to Tessellate shape [[51.6251554, 6.9730244] //...// [51.624605, 6.9736199] ]]. Possible malformed shape detected."
}
},
"status": 400
}
The actual shape looks like this:
Although this shape is quite weird (possibly comming from a bug in my own import scripts), it seems to be a valid geojson (passes the mapbox linter).
I can't seem to find something in the release notes or the github issues that would suggest that It would not be supported anymore by the new geo_shape field. This shape in particular is extracted from openstreetmap see here.
Therefore I'm wondering if this is an actual bug or if it's to be expected.
I index the whole planet-osm data and this issue affects roughly 14% documents (~11M docs).
