Enrich policy access "_id" field

Elastic Stack Elasticsearch
1

I am trying to use an enrich policy to match on a filename and return the _id fields of that doc to add in as a field in my secondary index. I have two indexes the first has 1 entry with field 'filename'. The second index uses a pipeline and enrich policy shown below to match on the filename and return the _id value of the first index entry as a field in the second index. This does not work the _id value is not returned as expected.

Any clues as to why this is?

{
    "description": "x",
    "processors": [
      {
        "set": {
          "field": "test",
          "value": "null"
        }
      },
      {
        "enrich": {
          "policy_name": "enrich-policy",
          "field": "filename",
          "target_field": "tmp",
          "max_matches": "1",
          "ignore_failure": true
        }
      },
      {
        "set": {
          "if": "ctx.containsKey('tmp')",
          "field": "test",
          "value": "{{tmp._id}}",
          "on_failure":[{
            "set":{
              "field":"test",
              "value":"!Issue setting field value!"
            }
          }]
        }
      },
      {
        "remove": {
          "field": ["tmp"],
          "ignore_failure": true
        }
      }
    ],
    "version": 1
  }

enrich-policy.json

{
  "match": {
    "indices": ["index-1"],
    "match_field": "filename",
    "enrich_fields": ["_id"]
  }
}
2

It seems the _ fields are restricted and cannot be accessed

' The _id field is restricted from use in aggregations, sorting, and scripting. In case sorting or aggregating on the _id field is required, it is advised to duplicate the content of the _id field into another field that has doc_values enabled.' (_id field | Elasticsearch Guide [7.12] | Elastic)

It talks about creating a copy into a new field that can be accessed but as described in the issue below _id is not accessible on ingest in the pipeline and is only assigned after.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.