Ingest pipeline with enrich policy that matches by _id - not supported?


I'd like to deduplicate data. We're hashing a subset of fields of the events into the document identifier. To do this in an ingest pipeline, I'd like to have an enrich policy like so:

    PUT /_enrich/policy/event-duplicates
      "match": {
        "indices": "events",
        "match_field": "_id",
        "enrich_fields": ["status"]

This gives

Could not traverse mapping to field [_id]. Could not find the [_id] field under [root]

Is it possible to match by _id?

Thanks, Dan.

I do not think this approach will work as indexed data will take some time to become available, but I do not see why you would need this in the first place anyway. If the document with the ID already exists it will be updated and result in no duplicates.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.