Migrating geo_shape data from 6.5.3 to 6.8.1: mapper [xxx] of different type, current_type [geo_shape], merged_type [geo_shape]

When migrating documents from ES 6.5.3 to 6.8.1 we are facing the following issue with the geo_shape type. By querying an indexed document from ES 6.5.3 with the following query on ES 6.8.1

{
  "size": 2000,
  "query": {
    "bool": {
      "filter": [
        {
          "geo_shape": {
            "stopovers_cloud": {
              "shape": {
                "type": "circle",
                "radius": "14006.0m",
                "coordinates": [
                  0.182766,
                  49.284982
                ]
              },
              "relation": "intersects"
            },
            "ignore_unmapped": false,
            "boost": 1
          }
        },
        {
          "geo_shape": {
            "stopovers_cloud": {
              "shape": {
                "type": "circle",
                "radius": "14006.0m",
                "coordinates": [
                  -1.542045,
                  48.977566
                ]
              },
              "relation": "intersects"
            },
            "ignore_unmapped": false,
            "boost": 1
          }
        },
        {
          "range": {
            "max_distance_as_the_crow_flies_in_meters": {
              "from": 102051,
              "to": null,
              "include_lower": true,
              "include_upper": true,
              "boost": 1
            }
          }
        },
        {
          "range": {
            "date": {
              "from": null,
              "to": "2019-09-01T12:00:00.000Z",
              "include_lower": true,
              "include_upper": true,
              "boost": 1
            }
          }
        },
        {
          "range": {
            "last_departure_date": {
              "from": "2019-09-01T04:00:00.000Z",
              "to": null,
              "include_lower": true,
              "include_upper": true,
              "boost": 1
            }
          }
        },
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "sort": [
    {
      "date": {
        "order": "asc"
      }
    }
  ]
}

It returns:

{
    "took": 3,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 3,
        "skipped": 0,
        "failed": 2,
        "failures": [
            {
                "shard": 0,
                "index": "offers_future_dev",
                "node": "uTW70IOqTdGGKE1n0-LPTA",
                "reason": {
                    "type": "query_shard_exception",
                    "reason": "failed to create query: THE_QUERY_ABOVE",
                    "index_uuid": "rW6MdkgiSr6zhOXgQZnwRg",
                    "index": "offers_future_dev",
                    "caused_by": {
                        "type": "unsupported_operation_exception",
                        "reason": "CIRCLE geometry is not supported"
                    }
                }
            }
        ]
    },
    "hits": {
        "total": 0,
        "max_score": null,
        "hits": []
    }
}

By reading a past thread here in which the person faced the same problem I have tried to update the mapping with the suggestion by @Ignacio_Vera:

curl -X PUT \
  http://localhost:19200/offers_future_dev/offer/_mapping \
  -H 'Content-Type: application/json' \
  -H 'cache-control: no-cache' \
  -d '{
    "properties": {
        "polyline": {
            "type": "geo_shape",
            "strategy" : "recursive"
        },
        "stopovers_cloud": {
            "type": "geo_shape",
            "strategy" : "recursive"
        }
    }
}'

but it results in the following error:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "mapper [stopovers_cloud] of different type, current_type [geo_shape], merged_type [geo_shape]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "mapper [stopovers_cloud] of different type, current_type [geo_shape], merged_type [geo_shape]"
  },
  "status": 400
}

The message is meaningless as it mentions "different type", which are actually the same: "current_type [geo_shape], merged_type [geo_shape]", i.e., geo_shape for both.

Given this situation we are now stuck on ES 6.5.3 as this breaking change between this version and 6.8 does not seem to be easy to solve. Any ideas are welcome.

Thanks in advance.

Ps.: For information supplementary details are on the comments as there is a 7K limit on the question body.

Our document:

{
  "id" : "1505458256",
  "revision_number" : 3,
  "comfort" : true,
  "viaggio_rosa" : false,
  "date" : "2019-09-01T11:40:00.000Z",
  "last_departure_date" : "2019-09-01T12:57:00.000Z",
  "last_departure_date_in_utc" : "2019-09-01T10:57:00.000Z",
  "freeway" : true,
  "bearing_range" : {
    "lte" : 278,
    "gte" : 237
  },
  "max_distance_as_the_crow_flies_in_meters" : 131509,
  "nb_seats" : 2,
  "booking" : {
    "auto_accept" : false,
    "mode" : 1,
    "corridoring_pending_requests" : 0,
    "corridoring_rejected_requests" : 0,
    "corridoring_accepted_requests" : 0
  },
  "last_update" : "2019-09-06T17:28:53.346Z",
  "has_routing_polyline" : true,
  "carrier_type" : "CARPOOLING",
  "polyline" : {
    "type" : "linestring",
    "coordinates" : [
      [
        0.182766,
        49.284982
      ],
      [
        0.18283,
        49.28494
      ],
      [
        0.20041,
        49.29223
      ],
      [
        0.19111,
        49.27509
      ],
      [
        0.15917,
        49.26389
      ],
      [
        0.08812,
        49.25685
      ],
      [
        0.04856,
        49.24572
      ],
      [
        -0.00243,
        49.24752
      ],
      [
        -0.10557,
        49.22305
      ],
      [
        -0.17439,
        49.20192
      ],
      [
        -0.2156,
        49.17052
      ],
      [
        -0.29976,
        49.17027
      ],
      [
        -0.3217,
        49.14829
      ],
      [
        -0.35712,
        49.14726
      ],
      [
        -0.37271,
        49.1364
      ],
      [
        -0.40703,
        49.14507
      ],
      [
        -0.4276,
        49.16569
      ],
      [
        -0.53714,
        49.14262
      ],
      [
        -0.63174,
        49.0914
      ],
      [
        -0.65093,
        49.07038
      ],
      [
        -0.71249,
        49.05091
      ],
      [
        -0.78209,
        49.04712
      ],
      [
        -0.85076,
        49.01551
      ],
      [
        -0.89938,
        49.00831
      ],
      [
        -1.013,
        48.94871
      ],
      [
        -1.0683,
        48.93684
      ],
      [
        -1.11708,
        48.89826
      ],
      [
        -1.20414,
        48.85958
      ],
      [
        -1.20105,
        48.87846
      ],
      [
        -1.2507,
        48.87284
      ],
      [
        -1.29767,
        48.89849
      ],
      [
        -1.33414,
        48.91012
      ],
      [
        -1.34939,
        48.90753
      ],
      [
        -1.4016,
        48.96474
      ],
      [
        -1.4286,
        48.95976
      ],
      [
        -1.50149,
        48.97615
      ],
      [
        -1.52088,
        48.9888
      ],
      [
        -1.54208,
        48.97752
      ],
      [
        -1.542045,
        48.977566
      ]
    ]
  },
  "stopovers" : [
    {
      "id" : null,
      "name" : null,
      "english_name" : null,
      "address" : "Pont-l'Évêque, France",
      "city" : "Pont-l'Évêque",
      "country" : "FR",
      "coordinate" : "49.284982,0.182766",
      "locale" : null,
      "is_precise" : true,
      "dropoff_date" : "2019-09-01T11:40:00.000Z",
      "pickup_date" : "2019-09-01T11:40:00.000Z",
      "timezone" : "Europe/Paris"
    },
    {
      "id" : null,
      "name" : null,
      "english_name" : null,
      "address" : "Hauteville-sur-Mer, France",
      "city" : "Hauteville-sur-Mer",
      "country" : "FR",
      "coordinate" : "48.977566,-1.542045",
      "locale" : null,
      "is_precise" : true,
      "dropoff_date" : "2019-09-01T13:30:00.000Z",
      "pickup_date" : "2019-09-01T13:30:00.000Z",
      "timezone" : "Europe/Paris"
    }
  ],
  "stopovers_cloud" : {
    "type" : "multipoint",
    "coordinates" : [
      [
        0.182766,
        49.284982
      ],
      [
        -1.542045,
        48.977566
      ]
    ]
  },
  "segments" : [
    {
      "trip_id" : 8214031260,
      "trip_permanent_id" : "1505458256-pont-l-eveque-hauteville-sur-mer",
      "start_stopover_index" : 0,
      "end_stopover_index" : 1,
      "booking_number" : 0,
      "seats_left" : 2,
      "distance_in_meters" : 161614,
      "duration_osrm_in_seconds" : 6982,
      "price" : {
        "value" : 9.0,
        "value_in_euro" : 9.0,
        "suggested_value" : 9.0,
        "currency" : "EUR"
      }
    }
  ]
}

use this mapping:

{
    "offer": {
        "dynamic": "strict",
        "properties": {
            "revision_number": {
                "type": "integer"
            },
            "bearing_range": {
                "type": "integer_range"
            },
            "carrier_type": {
                "type": "keyword"
            },
            "booking": {
                "type": "object",
                "properties": {
                    "auto_accept": {
                        "type": "boolean"
                    },
                    "mode": {
                        "type": "integer"
                    },
                    "corridoring_pending_requests": {
                        "enabled": false
                    },
                    "corridoring_rejected_requests": {
                        "enabled": false
                    },
                    "corridoring_accepted_requests": {
                        "enabled": false
                    }
                }
            },
            "comfort": {
                "type": "boolean"
            },
            "date": {
                "format": "date_time",
                "type": "date"
            },
            "last_departure_date": {
              "format": "date_time",
              "type": "date"
            },
            "last_departure_date_in_utc": {
              "format": "date_time",
              "type": "date"
            },
            "last_update": {
              "format": "date_time",
              "type": "date"
            },
            "max_distance_as_the_crow_flies_in_meters": {
                "type": "long"
            },
            "freeway": {
                "type": "boolean"
            },
            "id": {
                "type": "keyword"
            },
            "nb_seats": {
                "type": "integer"
            },
            "has_routing_polyline": {
                "type": "boolean"
            },
            "polyline": {
                "type": "geo_shape",
                "distance_error_pct": 0.10
            },
            "segments": {
                "type": "object",
                "enabled": false,
                "properties": {
                    "trip_id": {
                        "type": "long"
                    },
                    "trip_permanent_id": {
                        "type": "keyword"
                    },
                    "start_stopover_index": {
                        "type": "integer"
                    },
                    "end_stopover_index": {
                        "type": "integer"
                    },
                    "booking_number": {
                        "type": "integer"
                    },
                    "seats_left": {
                        "type": "integer"
                    },
                    "price": {
                        "type": "object",
                        "properties": {
                            "value": {
                                "type": "double"
                            },
                            "value_in_euro": {
                                "type": "double"
                            },
                            "suggested_value": {
                                "type": "double"
                            },
                            "currency": {
                                "type": "keyword"
                            }
                        }
                    },
                    "distance_in_meters": {
                      "type": "long"
                    },
                    "duration_osrm_in_seconds": {
                      "type": "integer"
                    }
                }
            },
            "stopovers": {
                "type": "object",
                "enabled": false,
                "properties": {
                    "id": {
                        "type": "long"
                    },
                    "name": {
                        "type": "keyword"
                    },
                    "english_name": {
                        "type": "keyword"
                    },
                    "address": {
                        "type": "keyword"
                    },
                    "city": {
                        "type": "keyword"
                    },
                    "country": {
                        "type": "keyword"
                    },
                    "coordinate": {
                        "type": "geo_point"
                    },
                    "locale": {
                        "type": "keyword"
                    },
                    "is_precise": {
                        "type": "boolean"
                    },
                    "pickup_date": {
                        "format": "date_time",
                        "type": "date"
                    },
                    "dropoff_date": {
                        "format": "date_time",
                        "type": "date"
                    },
                    "timezone": {
                        "type": "keyword"
                    }
                }
            },
            "stopovers_cloud": {
                "type": "geo_shape"
            },
            "viaggio_rosa": {
                "type": "boolean"
            }
        }
    }
}

Hi Eduardo,

Regarding your issue you cannot just update the mapping. You need to reindex your documents in a new index. When you create the mapping on the new index, you need to make sure you add the strategy option to your geo_shape fields. You can probably use the reindex API for that task.

I think the error message is misleading and I will have a closer look.

1 Like

Thanks, that indeed fixed the problem. The issue was to reindex then try to update the mapping. By creating the maping already with the "strategy": "recursive" then indexing the document worked nicely.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.