Ingest pipeline with enrich policy that matches by _id - not supported?

Elastic Stack Elasticsearch
1

Hi,

I'd like to deduplicate data. We're hashing a subset of fields of the events into the document identifier. To do this in an ingest pipeline, I'd like to have an enrich policy like so:

    PUT /_enrich/policy/event-duplicates
    {
      "match": {
        "indices": "events",
        "match_field": "_id",
        "enrich_fields": ["status"]
      }
    }

This gives

Could not traverse mapping to field [_id]. Could not find the [_id] field under [root]

Is it possible to match by _id?

Thanks, Dan.

2

I do not think this approach will work as indexed data will take some time to become available, but I do not see why you would need this in the first place anyway. If the document with the ID already exists it will be updated and result in no duplicates.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.