Geo-polygon query inside ingest pipeline

[Roy_Levy]

Hey,

I want to use the Geo-polygon query (Geo-polygon query | Elasticsearch Guide [7.13] | Elastic) inside an ingest pipeline and I've been thinking how to do it - as there is no query processor and the example for the geo polygon is with this type of query.

The reason for me trying to use this tool is to enrich documents which's country location is null, by checking their lat and lon fields and determining which country the doc is from.

Is there anyway this could be done in a pipeline?

[stephenb (Stephen Brown)]

I am thinking this is exactly what you are looking for?

Example: Enrich your data based on geolocation | Elasticsearch Guide [7.13] |...

[Roy_Levy]

It seems like the way to go for me, but when I'm trying this example I have too many coordinates in my Polygon shape that it does not fit in the queries in the DevTools. Is there any way to get the max query size up in DevTools?

[stephenb (Stephen Brown)]

Perhaps just import a GEO JSON file from the Maps App.
That might be easier.

[Roy_Levy]

Ohhh that's a cool feature I didn't know of!
Thanks!

[Roy_Levy]

Hey, I'm really close to having this all figured out - I'm creating a pipeline with an enrichment policy like you suggested and it works, but the coordinates used in the policy are very big.

When I'm getting a document through the pipeline the new enriched document has the fields I've specified in enrich_fields , but it also has the coordinates used to make the comparison. Is there any way to not add the coordinates field to every pipelined document?

This is the policy that I'm using:

PUT /_enrich/policy/county_and_state_policy
{
  "geo_match": {
    "indices": ["county_index1", "county_index2"],
    "match_field": "coordinates",
    "enrich_fields": [ "county_name", "state_name" ]
  }
}

[stephenb (Stephen Brown)]

Are you doing point in polygon match or polygon intersects match?

What does the actual enrich processor look like / take as arguments?

I think the match field is always returned.

The target_field used to store appended enrich data for incoming documents. This field contains the match_field and enrich_fields specified in your enrich policy.

You can always do a drop processor afterwards to get rid of that field.

[Roy_Levy]

I'm doing polygon intersects match with coordinates and lat, lon as input for the intersection.
Yeah seems like there is no way to exclude it, so I think I'll go with your option to get rid of it afterwards.

Thank you very much for the help!

[stephenb (Stephen Brown)]

If you want to open an issue in GitHub please do let me know if you do and I'll tag onto it

elastic/elasticsearch

[Roy_Levy]

Hey Stephen,

I've created the feature issue in GitHub: Excluding match_field from geolocation enrich policy · Issue #74001 · elastic/elasticsearch · GitHub

Thanks!