Geo_shape query point in polygon runtime field for pre-indexed docs

Elastic Stack Kibana
1

I'm trying to create a geo_shape query that will be used in a runtime field to tag a polygon 'id/name' to a point that it contains. I have two pre-indexed indexes: one for neighborhoods (polygon) and one for car crashes (point) and would like the car crash index to be tagged via a runtime query field with the neighborhood name that it is located in. - so every new crash doc gets its neighborhood associated with it.

I feel that I'm close to getting it, but still getting errors after trying for a long while, I thought I reach out for help. Below is the query that I'm testing with. 'neighs' = polygon index for neighborhoods and 'crash' = point index. It returns a Illegal argument exception, Required [type] error. I've tried to add a type in the query in various ways (point, _doc, polygon, etc) to no avail.
The geometry/coord fields for neighs is called: 'geometry' and 'location' for the crash point coords.

Thanks for your help!

GET neighs/_search
{
  "query": {
    "bool": {
      "filter": {
        "geo_shape": {
    
          "location": {
            
            "indexed_shape": {
              "index": "crash",
              "type": "point",
              "id": "ADI497001X",
              "path": "location"
             
            }

          }
        }
      }
    }
  }
}

Here's an example of a neighborhood indexed doc:

{
        "_index": "neighs",
        "_id": "gcy52oQBjjcu_kHT5h28",
        "_score": 1,
        "_source": {
          "Shape_STAr": 0.0000197467779799,
          "Population": 889,
          "geometry": {
            "coordinates": [
              [
                [
                  -76.60941,
                  39.325561
                ],
                [
                  -76.609407,
                  39.325799
                ],
              etc......
            ],
            "type": "Polygon"
          },
   
          "Name": "Abell"
        }
      },

And here is a 'crash' indexed doc:

{
        "_index": "crash",
        "_id": "ADH56700B2",
        "_score": 1,
        "_ignored": [
          "narrative.keyword"
        ],
        "_source": {
          "@timestamp": "2023-04-06T18:00:12.026635Z",
          "crashdate": "2023-03-29T04:00:00Z",
          "location": {
            "lon": -76.6407616882842,
            "lat": 39.3155478455349
          },
          "reporttype_type": "Property Damage : Vehicle",
          "source": "ACRS",
          "type": "Vehicle"
        }
      },
2

Hi @bchranko
I edited you post an formatted the code because it was pretty much unreadable... You can format code by selecting the code block and pressing the </> button

With respect to the question it is not clear to me what try to do..

It sounds like you're trying to enrich you're crash data with the neighborhood... Not sure run time is the correct way to do that but perhaps.

Generally That would be done with an enrich processor using the geometry.

3

Thanks Stephen. Thank you for formatting the code. I'm not sure I can be clearer in what I'm looking for when needing to tag a geo_point index with attributes of a geo_polygon index (assigning a neighborhood (polygon) name to a point (crash location) based on containment). but it looks like you managed to decipher it, and I will look further into the recommendation for a enrich processor using geometry. It just seemed like a lot to do just for updating one field based on its location, so I was seeing if it can be handled more efficiently with a runtime field. Thanks so much for your help!

4

Hi @bchranko

Yes now I understand what you are trying to do is a fairly simple concept, trying a geospatial query in a runtime field threw me off.

However Today

elasticsearch does not natively support joins

Runtime lookup type is a very simple term query (think exact match) to lookup fields from another index. This a very initial step (why experimental) in to join-like features

Runtime lookup does not support geospatial queries.. geospatial contains / intersects are not "simple look ups"

I can see the use case you certainly can create a feature request against the elasticsearch repo...

Runtime fields by their nature need to be fast / lightweight or they will not scale.. so not sure if / how that would work.

Today enrich is the correct method if you want the field "tag" to be available for any operation...

Now if you just want to display on a map... The map actually supports what you want on the map but it will not be stored in a runtime field.

See here for that.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.