# Enrich data using ingest pipeline and enrich policy override data

**URL:** https://discuss.elastic.co/t/enrich-data-using-ingest-pipeline-and-enrich-policy-override-data/316049
**Category:** Elasticsearch
**Tags:** ingest-pipeline
**Created:** [October 7, 2022, 9:07am UTC](https://discuss.elastic.co/t/enrich-data-using-ingest-pipeline-and-enrich-policy-override-data/316049 "2022-10-07T09:07:34Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Fariddev](https://avatars.discourse-cdn.com/v4/letter/f/82dd89/32.png) [@Fariddev](https://discuss.elastic.co/u/Fariddev)
#### Post date: [October 7, 2022, 9:07am UTC](https://discuss.elastic.co/t/enrich-data-using-ingest-pipeline-and-enrich-policy-override-data/316049/1 "2022-10-07T09:07:34Z")

</div>

Hi,  
I want to enrich data in an index (add nationality of authors in books index) using data in another index (authors). I use an ingest pipeline with an enrich policy.

Here is a simple example :

Indices

```auto
PUT books
{
  "mappings": {
    "properties": {
      "title": {
        "type": "keyword"
      },
      "authors": {
        "type": "nested",
        "properties": {
          "code": {
            "type": "keyword"
          },
          "name": {
            "type": "keyword"
          },
          "page_written": {
            "type": "integer"
          },
          "nationality": {
            "type": "keyword"
          }
        }
      }
    }
  }
}

PUT authors
{
  "mappings": {
    "properties": {
      "code": {
        "type": "keyword"
      },
      "name": {
        "type": "keyword"
      },
      "nationality": {
        "type": "keyword"
      }
    }
  }
}

```

Some authors :

```auto
PUT authors/_doc/a1
{
  "code": "a1",
  "name": "John",
  "nationality": "french"
}

PUT authors/_doc/a2
{
  "code": "a2",
  "name": "Jack",
  "nationality": "english"
}

```

Policy and pipeline :

```auto
PUT /_enrich/policy/enrich_author_policy
{
  "match": {
    "indices": "authors",
    "match_field": "code",
    "enrich_fields": [
      "nationality"
    ]
  }
}

POST /_enrich/policy/enrich_author_policy/_execute

PUT /_ingest/pipeline/enrich_author_pipeline
{
  "processors": [
    {
      "foreach": {
        "field": "authors",
        "processor": {
          "enrich": {
            "policy_name": "enrich_author_policy",
            "field": "_ingest._value.code",
            "target_field": "_ingest._value"
          }
        }
      }
    }
  ]
}

```

Then I add a books using my pipeline :

```auto
PUT books/_doc/b1?pipeline=enrich_author_pipeline
{
  "title": "Programming 101",
  "authors": [
    {
      "code": "a1",
      "name": "John",
      "page_written": 120
    },
    {
      "code": "a2",
      "name": "Jack",
      "page_written": 113
    }
  ]
}

```

Nationality for each authors is retrieve but the data in books index is overriden instead of enriched.

Result :

```auto
{
  "title" : "Programming 101",
  "authors" : [
	{
	  "nationality" : "french",
	  "code" : "a1"
	},
	{
	  "nationality" : "english",
	  "code" : "a2"
	}
  ]
}

```

Expected result :

```auto
{
  "title" : "Programming 101",
  "authors" : [
	{
      "code": "a1",
      "name": "John",
      "page_written": 120,
	  "nationality" : "french"
    },
    {
      "code": "a2",
      "name": "Jack",
      "page_written": 113,
	  "nationality" : "english"
    }
  ]
}

```

Is there a way to not override existing data ?  
I know I can add field like the example in documentation, but I got a weird result with duplicate information :

```auto
{
  "title" : "Programming 101",
  "authors" : [
	{
      "code": "a1",
      "name": "John",
      "page_written": 120,
	  "infos": {
		"code": "a1",
		"nationality" : "french"
	  }
    },
    {
      "code": "a2",
      "name": "Jack",
      "page_written": 113,
	  "infos": {
		"code": "a1",
		"nationality" : "english"
	  }
    }
  ]
}

```

Thanks.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 4, 2022, 9:08am UTC](https://discuss.elastic.co/t/enrich-data-using-ingest-pipeline-and-enrich-policy-override-data/316049/2 "2022-11-04T09:08:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.