Creating a Geo Enrich Policy for existing Index, encountering error when adding the enriched field to existing documents by Reindexing

Elastic Stack Elasticsearch
,
1

Hi there!

I'm trying to create an Enrich Policy using geo_match for the first time and wanted to test it by applying it to an existing index and documents within it. I've added the field that I intend to populate with the enriched data in the existing Index mapping:

PUT /cases_example/_mapping
{
  "properties": {
    "WWTParea": {
      "type": "text"
    }
  }
}

I've created an Enrich Policy where I check the geometry (Geo-shape) field which holds the bounding coordinates of polygons and assigns the text name called 'WWTP' as the enriched data:

PUT /_enrich/policy/in_wwtp_policy
{
  "geo_match": {
    "indices": "servingareas",
    "match_field": "geometry",
    "enrich_fields": ["WWTP"]
  }
}

I created an Ingest Pipeline that uses this policy to check where Geo-point data in the incoming documents (location) intersects (have also tried WITHIN - this results in a Bad Gateway error) the polygons defined in the Geo Enrich Policy, assigning the enriched text data to the newly-created "WWTParea" property:

PUT /_ingest/pipeline/wwtp_lookup
{
  "processors": [
    {
      "enrich": {
        "description": "Add WWTP area name that case falls within based on geolocation",
        "policy_name": "in_wwtp_policy",
        "field": "location",
        "target_field": "WWTParea",
        "shape_relation": "INTERSECTS"
      }
    }
  ]
}

I'm trying to test the pipeline by applying it to a Reindex of my existing Documents:

POST _reindex
{
  "source": {
    "index": "cases_example"
  },
  "dest": {
    "index": "temporary_ww_index",
    "pipeline": "wwtp_lookup"
  }
}

But I receive the following error:

"index": "temporary_ww_index",
      "cause": {
        "type": "illegal_argument_exception",
        "reason": "mapper [WWTParea.geometry.coordinates] cannot be changed from type [dense_vector] to [float]"
      },
      "status": 400

I'm confused because the data that should be enriched ("WWTP") is coming from a text field and this is the only field I am trying to copy into the new property ("WWTParea") via the enriched policy. Is there something I am missing in how the Policy/Pipeline is trying to write the data from "servingareas" to the "cases_example" documents based on the geo_match? Any insight would be appreciated!

2

Heya @jlogan

The mapping for temporary_ww_index didn't have a value set for WWTParea.geometry.coordinates and it tried to dynamically set it to the best value it could (for many float[] values we set it to be a vector).

You should set the mapping for temporary_ww_index statically before trying to reindex.

1 Like
3

Thanks @BenTrent! I also realized that it was trying to retain the coordinates as part of the enrich policy call after reading this Elastic blog post. I tried revising the processor to prevent it from retaining a value from the geometry and keeping only the text variable I'm interested in:

PUT /_ingest/pipeline/wwtp_lookup
{
  "processors": [
    {
      "enrich": {
        "description": "Add WWTP area name that case falls within based on geolocation",
        "policy_name": "in_wwtp_policy",
        "field": "location",
        "target_field": "WWTParea",
        "shape_relation": "INTERSECTS"
      }
    },
    {
      "remove": {
        "field": "WWTParea.geometry.coordinates",
        "description": "Remove the coordinate field so it does not append to enriched data field"
      }
    }
  ]
}

The problem now is that when I call the Pipeline in console, it processes for a while and then throws a 502 error code for Bad Gateway. Not sure why, since I'm running it directly in the dev_tools console. Appreciate the input!