# Configuration of transform data with custom pipeline

**URL:** <https://discuss.elastic.co/t/configuration-of-transform-data-with-custom-pipeline/335923>\
**Category:** APM\
**Tags:** ui\
**Created:** [June 14, 2023, 2:50am UTC](https://discuss.elastic.co/t/configuration-of-transform-data-with-custom-pipeline/335923 "2023-06-14T02:50:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![JasonREC](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JasonREC](https://discuss.elastic.co/u/JasonREC)\
**Post date:** [June 14, 2023, 2:50am UTC](https://discuss.elastic.co/t/configuration-of-transform-data-with-custom-pipeline/335923/1 "2023-06-14T02:50:27Z")

</div>

Hi all,

I am currenly using Elastic APM to monitor my application. and I noticed that transaction log data that created by APM agent is automactilly store in an Elasticsearch index. I want to enrich every doucment before it write into the index. And, I found a offical tutorial document showing me how to do. but I cannot find the configuation option in my UI.

Tutorial Link:[Tutorial: Transform data with custom ingest pipelines | Fleet and Elastic Agent Guide [8.7] | Elastic](https://www.elastic.co/guide/en/fleet/8.7/data-streams-pipeline-tutorial.html)  
Elastic Stack version 8.8  
Currently, I have step 1 done, and stuck on step 2 adding the injest pipeline

 ![step2](https://us1.discourse-cdn.com/elastic/original/3X/2/1/21c81ac33fec0f4bbff0b5a16a739760333bf75d.png)  
I cannot find the data stream to add the injestpipeline

---

<div class="post-metadata">

**Author:** ![bmorelli25](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bmorelli25/32/38738_2.png) [@bmorelli25](https://discuss.elastic.co/u/bmorelli25)\
**Post date:** [June 14, 2023, 3:10am UTC](https://discuss.elastic.co/t/configuration-of-transform-data-with-custom-pipeline/335923/2 "2023-06-14T03:10:33Z")

</div>

Hi, @JasonREC.

That tutorial is designed for the System integration. Take a look at our [APM ingest pipelines](https://www.elastic.co/guide/en/apm/guide/current/ingest-pipelines.html) docs instead. Then try following the steps in [this custom APM ingest pipeline tutorial](https://www.elastic.co/guide/en/apm/guide/current/custom-filter.html#filters-ingest-pipeline).

---

<div class="post-metadata">

**Author:** ![JasonREC](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@JasonREC](https://discuss.elastic.co/u/JasonREC)\
**Post date:** [June 14, 2023, 8:14am UTC](https://discuss.elastic.co/t/configuration-of-transform-data-with-custom-pipeline/335923/3 "2023-06-14T08:14:49Z")

</div>

Hi, @bmorelli25 ,

Thanks for the reply!

I read both the document you provided and set up the pipeline for testing like below:

 ![pipeline2](https://us1.discourse-cdn.com/elastic/original/3X/3/7/37a964dbabeac44947a41b3029e1d70a55f4ecca.png)

The above Injest Pipeline with an append processor can sucessfully show the upcoming data with field "processor\_ test" and the value

but then, I try to use the enrich processor, but it does not work somehow

Here is my set up,  
source Index:

 ![source-index](https://us1.discourse-cdn.com/elastic/original/3X/5/5/55bd78f9ecce13925a3811b490450c3fbe0b4b40.png)

policy:

 ![policy](https://us1.discourse-cdn.com/elastic/original/3X/a/b/abb0ed7c1877a0c7335e8e9a67c50f1d324cd123.png)

enrich processor"

 ![injestpipeline](https://us1.discourse-cdn.com/elastic/original/3X/3/e/3e46bdda11079b452507fbf6e87730c60c5722c6.png)

Lastly I have applied the injest pipeline to the APM custom pipeline

```auto
PUT _ingest/pipeline/traces-apm@custom
{
  "processors": [
    {
      "pipeline": {
        "name": "limit_lookup" 
      }
    }
  ]
}

```

Basically, as long as the upcoming document that contains the labels\_data\_name is matched the labels\_database\_name that in the source index. I would like it to add that couple fields into the document before it write into index.

Here is my apm index data looks like, the upcoming data have a field called labels\_database\_name like below

 ![kibana](https://us1.discourse-cdn.com/elastic/original/3X/d/4/d47137bd8f94200408c26f8eab7c090d47f90f51.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 12, 2023, 8:14am UTC](https://discuss.elastic.co/t/configuration-of-transform-data-with-custom-pipeline/335923/4 "2023-07-12T08:14:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.