# APM custom pipeline with enrich processor does not work

**URL:** <https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026>\
**Category:** Elasticsearch\
**Tags:** ingest-pipeline\
**Created:** [June 15, 2023, 1:56am UTC](https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026 "2023-06-15T01:56:32Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![JasonREC](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JasonREC](https://discuss.elastic.co/u/JasonREC)\
**Post date:** [June 15, 2023, 1:56am UTC](https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026/1 "2023-06-15T01:56:33Z")

</div>

Hi all, I am trying to use enrich processor in the injest pipeline and apply into apm custom pipeline, but it does not work.

Here is my set up,  
source index:

 ![source-index](https://us1.discourse-cdn.com/elastic/original/3X/5/5/55bd78f9ecce13925a3811b490450c3fbe0b4b40.png)

Enrich process policy:

 ![policy](https://us1.discourse-cdn.com/elastic/original/3X/a/b/abb0ed7c1877a0c7335e8e9a67c50f1d324cd123.png)

Injest pipeline name: limit\_lookup with enrich processor:

 ![injestpipeline](https://us1.discourse-cdn.com/elastic/original/3X/3/e/3e46bdda11079b452507fbf6e87730c60c5722c6.png)

And, I have applied this to my to apm custom pipeline

```auto
PUT _ingest/pipeline/traces-apm@custom
{
  "processors": [
    {
      "pipeline": {
        "name": "limit_lookup" 
      }
    }
  ]
}

```

Basically, as long as the incoming document that contains the labels\_data\_name is matched the labels\_database\_name that in the source index. I would like it to add that couple fields into the document before it write into index.

Here is my apm index data looks like, the incoming data have a field called labels\_database\_name like below:

 ![kibana](https://us1.discourse-cdn.com/elastic/original/3X/d/4/d47137bd8f94200408c26f8eab7c090d47f90f51.png)

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 15, 2023, 2:30am UTC](https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026/2 "2023-06-15T02:30:27Z")

</div>

Something is not adding up, you are referring to a field name `labels_database_name`, but the screenshot you shared from your kibana does not have this field, it has a field named `labels.database_name`, which is completely different.

Can you go into Discover on Kibana, expand a document and share the Json of how this document looks like?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 15, 2023, 2:35am UTC](https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026/3 "2023-06-15T02:35:50Z")

</div>

Oh I see.

In your source index your field is `labels_database_name`, but from what you shared from Kibana the field in your document is `labels.database_name`, which is different, you need to use this in your enrich processor.

Try to change the the `field` in your enrich processor to `labels.database_name`.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 15, 2023, 3:10am UTC](https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026/4 "2023-06-15T03:10:38Z")

</div>

Is this the same as [Configuration of transform data with custom pipeline - #3 by JasonREC](https://discuss.elastic.co/t/configuration-of-transform-data-with-custom-pipeline/335923/3)?

---

<div class="post-metadata">

**Author:** ![JasonREC](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JasonREC](https://discuss.elastic.co/u/JasonREC)\
**Post date:** [June 15, 2023, 4:36am UTC](https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026/5 "2023-06-15T04:36:29Z")

</div>

@leandrojmp  
Oh! I didn't realize that when I upload my csv data to elastic, it automactially change my field from labels.database\_name -\> labels\_database\_name . I look at this issue in other discussion it is due to the elastic dot conflict problem.

 ![file_content](https://us1.discourse-cdn.com/elastic/original/3X/2/c/2c538aa12b0311a3e47bad10c5b98883d7448812.png)  
 ![file_stat](https://us1.discourse-cdn.com/elastic/original/3X/d/c/dca31654db893f1bd60c9b762a4daf2c20a0b4c5.png)

so, I try to use update query to change the field name, but it failed due to Elastic consider the dot is implying I am accessing an object attribute instead of considering it is a part of name.

and then I try to add a rename processor before the enrich processor, so that I change the label.database\_name back to label\_database\_name. but seems does not work also.

---

<div class="post-metadata">

**Author:** ![JasonREC](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JasonREC](https://discuss.elastic.co/u/JasonREC)\
**Post date:** [June 15, 2023, 8:53am UTC](https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026/6 "2023-06-15T08:53:26Z")

</div>

Hi, @leandrojmp

Never mind, I just found out there is a over write setting which can help me change the labels\_database\_name back to labels.database\_name

Now, the enrich processor works as expected 😄  
Thank you

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 13, 2023, 8:53am UTC](https://discuss.elastic.co/t/apm-custom-pipeline-with-enrich-processor-does-not-work/336026/7 "2023-07-13T08:53:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
