1.7 to 5.3 Migration: Range filtering on geo_points not working on v5.3

Hi friends,

I'm migrating my Elasticsearch from 1.7 to 5.3, and have noticed inconsistencies in how range filters interact with geo_point data types. My query to 1.7 returns data successfully, while my query to 5.3 gives me nothing.

My search query to 1.7 (some fields redacted for company reasons) looks something like:

{
  "query": {
    "filtered": {
      "filter": {
        "and": {
          "filters": [
            {
              "bool": {
                "must": [
                  {
                    "and": {
                      "filters": [
                        {
                          "range": {
                            "x.y.z.lon": {
                              "from": null,
                              "to": -74.97070312500001,
                              "include_lower": true,
                              "include_upper": false
                            },
                            "_cache": false
                          }
                        }
                      ]
                    }
                  }
                ],
                "_cache": true
              }
            }
          ],
          "_cache": false
        }
      }
    }
  }
}

My query to 5.3 looks something like this:

{
  "from": 0,
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "bool": {
            "filter": [
              {
                "range": {
                  "x.y.z.lon": {
                    "gt": null,
                    "lt": "-74.97070312500001"
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }
}

Here are my mappings for the relevant fields (extra fields trimmed out):

v1.7:

"my_17_mapping": {
    "_routing": {
      "required": true
    },
    "dynamic": "true",
    "dynamic_templates": [
      {
        "strings_template": {
          "mapping": {
            "norms": {
              "enabled": false
            },
            "index": "not_analyzed",
            "type": "string",
            "doc_values": true
          },
          "match_mapping_type": "string",
          "match": "*"
        }
      },
      {
        "dynamic_default": {
          "mapping": {
            "norms": {
              "enabled": false
            },
            "doc_values": true
          },
          "match_mapping_type": "*",
          "match": "*"
        }
      }
    ],
    "date_detection": false,
    "properties": {
      "x": {
        "properties": {
          "y": {
            "properties": {
              "z": {
                "type": "geo_point",
                "lat_lon": true,
                "doc_values": true
              }
            }
          }
        }
      }
    }
  }
}

v5.3:

{
  "my_53_index": {
    "mappings": {
      "my_53_mapping": {
        "dynamic": "true",
        "_routing": {
          "required": true
        },
        "dynamic_templates": [
          {
            "strings_template": {
              "match": "*",
              "match_mapping_type": "string",
              "mapping": {
                "doc_values": true,
                "norms": false,
                "type": "keyword"
              }
            }
          },
          {
            "dynamic_default": {
              "match": "*",
              "mapping": {
                "doc_values": true,
                "norms": false
              }
            }
          }
        ],
        "date_detection": false,
        "numeric_detection": false,
        "properties": {
          "x": {
            "properties": {
              "y": {
                "properties": {
                  "z": {
                    "type": "geo_point"
                  },
                }
              }
            }
          }
        }
      }
    }
  }
}

Could anyone help me understand why these queries aren't returning the same thing? Perhaps a breaking change between the two versions or a problem with my mappings?

Hi,

I have no much experience with 1.7 but geo_point field uses a totally different structure in 5.3 and I think you cannot use range queries to filter them. Have a look to the documentation:

https://www.elastic.co/guide/en/elasticsearch/reference/5.3/geo-point.html

I think you need to use a geo_boundig_box for what you are trying to do, something like:

{
  "from": 0,
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "bool": {
            "filter": [
              {
                "geo_bounding_box": {
                  "x.y.z.lon": {
                   "top_left" : {
                            "lat" : 90,
                            "lon" : -180
                        },
                        "bottom_right" : {
                            "lat" : -90,
                            "lon" : -74.97070312500001
                        }
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }
}

Interesting, thanks!

Just one followup question. If geo_point fields do not support range filtering, how come no exceptions are thrown to let me know that this query is invalid?

Not sure why you are not getting an exception. In newer versions of Elasticsearch it does throw an error if you try to run a range query over a geo_point field:

"caused_by": {
            "type": "illegal_argument_exception",
            "reason": "Field [location] of type [geo_point] does not support range queries"
          }

Maybe an issue on the version you are working on but I have not tried it.

I revisited this issue and I understand why you do not get an error. The field you are using in the range aggregation is 'x.y.z.lon' which it does not exist in the new version. This is not an error in Elasticsearch, if you change the field to `x.y.z' then you will get the error above.

Gotcha, thanks!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.