Filtering/Querying by indexed geo_shapes

Elastic Stack Elasticsearch
1

Hi there,

I am attempting to filter or query a index with geo points by using a geo_shape I have indexed already. For context, the "ais_2016_nov_zone11_enriched5" index is the one with geo points and the "ben_clusters" index has geo_shapes.

The following is what I am attempting to query with:

GET /ais_2016_nov_zone11_enriched5/_search
  {
    "query": {
        "geo_shape": {
            "location": {
               "relation" : "within",
                "indexed_shape": {
                        "index" : "ben_clusters",
                        "id": "b6qJs2sBrzkYGJItEBqq",
                        "type" : "_doc",
                        "path": "location"
                    }
                }
            }
        }
  }

And this is the error message I am receiving:

{
  "error": {
    "root_cause": [
      {
        "type": "query_shard_exception",
        "reason": "failed to find geo_shape field [location]",
        "index_uuid": "CP5DfnHITaK83JS-rlidHA",
        "index": "ais_2016_nov_zone11_enriched5"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "ais_2016_nov_zone11_enriched5",
        "node": "XGYWOAm0TvCS_ZY6wAVE_w",
        "reason": {
          "type": "query_shard_exception",
          "reason": "failed to find geo_shape field [location]",
          "index_uuid": "CP5DfnHITaK83JS-rlidHA",
          "index": "ais_2016_nov_zone11_enriched5"
        }
      }
    ]
  },
  "status": 400
}

I was using this page for reference https://www.elastic.co/guide/en/elasticsearch/guide/1.x/indexed-geo-shapes.html

Any help would be appreciated. Also as a note, the geo_shapes are being stored in a field named "location", and my geo_points are being stored in a field named "GEO_POINT".

2

You are looking to very old documentation, could you tell us which version of Elasticsearch you are using?

3

I believe I am on version 7. By running GET / I receive the following in Dev Tools:

{
  "name" : "instance-0000000022",
  "cluster_name" : "de6583e6d4dc4032bb77e2ad6cf11d02",
  "cluster_uuid" : "o8LiGHC0QPOrALpcmWjVeg",
  "version" : {
    "number" : "7.0.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "e4efcb5",
    "build_date" : "2019-04-29T12:56:03.145736Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.7.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
4

You need to look into the documentation for your version:

https://www.elastic.co/guide/en/elasticsearch/reference/7.0/query-dsl-geo-shape-query.html

The geo_shape queries only work on geo_shape fields, therefore you cannot apply that query in a geo_point field.

That is what the error is telling you, it could not find a field called location of type geo_shape.

5

So from the documentation it seems like there is a point being stored as a geo_shape, so I would need to reindex my current data into the geo_shape type. Then run a filter for geoshape points inside the Polygon geoshapes, and then reindex them back into geo_points to visualize them on a coordinate map?

6

What you can do is to index them as both geo_point and geo_shape. Once you filter the documents using the geo_shape field you will have the geo_point field available if that is what you need to visualise it.

7

I'm having a bit of trouble reindexing my points as geoshapes. I've tried the following to reindex

POST _reindex
{
  "source": {
    "index": "EnrichedDataSet"
  },
  "dest": {
    "index": "DataSetForPointShapes"
  },
  "script": {
    "source": "ctx._source.Shape_Point = 'POINT (ctx._source.LAT ctx._source.LON)'"
  }
}

and

POST _reindex
{
  "source": {
    "index": "enrichedDataSet"
  },
  "dest": {
    "index": "DataSetForPointShapes"
  },
  "script": {
    "source": "ctx._source.Shape_Point",
        "location": {
        "type": "point",
        "coordinates" : "[ctx.LAT , ctx.LON]"
    }
  }
}


Getting errors of "ctx is not a number" parsing error, and "[script] unknown field [location], parser not found"

I was hoping you could maybe shed some light on the proper syntax as the only references I have are for specific point inputs, which I cannot do manually.

References: 

https://www.elastic.co/guide/en/elasticsearch/reference/current/geo-shape.html 

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-shape-query.html
8

Here is my example, I have the following source mapping with geo point:

PUT /source
{
 "mappings": {
   "properties": {
      "location": {
        "type": "geo_point"
     }
   }
 }
}

And here is the target with geo_shape:

PUT /target
{
 "mappings": {
   "properties": {
      "location": {
        "type": "geo_shape"
     }
   }
 }
}

To reindex from source to target I can do the following:

POST _reindex
{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "target"
  },
  "script": {
    "source": "ctx._source.location = 'POINT (' + ctx._source.location.lon  + ' ' + ctx._source.location.lat + ')'"
  }
}

** EDIT: Note the order of lat/lon