# APM Filter incoming data

**URL:** <https://discuss.elastic.co/t/apm-filter-incoming-data/275971>\
**Category:** APM\
**Tags:** java\
**Created:** [June 15, 2021, 9:51am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971 "2021-06-15T09:51:36Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![binschlag](https://avatars.discourse-cdn.com/v4/letter/b/a4c791/32.png) [@binschlag](https://discuss.elastic.co/u/binschlag)\
**Post date:** [June 15, 2021, 9:51am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/1 "2021-06-15T09:51:36Z")

</div>

Is there any possibility to filter out APM data (errors, specific urls, ...), so that apm-server does not index them to elasticsearch?

Or is the only way to set the apm-server output to logstash and drop unwanted data in logstash?

This is doable but I think this is a huge performance overhead, so directly filtering in apm-server would be prefered if possible

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [June 15, 2021, 10:15am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/2 "2021-06-15T10:15:21Z")

</div>

In the Java agent you can use the [`ignore_exceptions`](https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-ignore-exceptions) and the [`transaction_ignore_urls`](https://www.elastic.co/guide/en/apm/agent/java/current/config-http.html#config-transaction-ignore-urls) config options to achieve that.

Otherwise, you can use [Ingest pipelines](https://www.elastic.co/guide/en/elasticsearch/reference/7.13/ingest.html) with a [drop processor](https://www.elastic.co/guide/en/elasticsearch/reference/7.13/drop-processor.html) for that.

I hope this helps.

---

<div class="post-metadata">

**Author:** ![binschlag](https://avatars.discourse-cdn.com/v4/letter/b/a4c791/32.png) [@binschlag](https://discuss.elastic.co/u/binschlag)\
**Post date:** [June 15, 2021, 11:19am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/3 "2021-06-15T11:19:03Z")

</div>

Thanks @Eyal_Koren ... Didn't know this options, but this may do it.

I've tested ignore\_exceptions and this works fine, but have a question about "transaction\_ignore\_urls"  
I've set this:  
transaction\_ignore\_urls=/internal/state/health,/devops/alive

But e.g. [http://127.0.0.1/devops/alive](http://127.0.0.1/devops/alive) is still instrumented.

Do I have here any misconfiguration?

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [June 15, 2021, 12:14pm UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/4 "2021-06-15T12:14:54Z")

</div>

If you are matching against exactly `http://127.0.0.1/devops/alive`, then your configuration should work. If this is not the exact and full URL, then use a wildcard at the end of the pattern.  
Otherwise, please provide your exact configurations, a debug log and the full URL that should being matched and doesn't, maybe we can find why.

---

<div class="post-metadata">

**Author:** ![binschlag](https://avatars.discourse-cdn.com/v4/letter/b/a4c791/32.png) [@binschlag](https://discuss.elastic.co/u/binschlag)\
**Post date:** [June 16, 2021, 6:10am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/5 "2021-06-16T06:10:56Z")

</div>

url.full=http://127.0.0.1/devops/alive

in my elasticapm.properties i have set:  
transaction\_ignore\_urls=http://127.0.0.1/devops/alive

But still find entries in my apm-indices with this url.

We have set the following configuration - parameters:

- application\_packages
- service\_name
- environment
- server\_urls

---

<div class="post-metadata">

**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\
**Post date:** [June 16, 2021, 7:54am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/6 "2021-06-16T07:54:23Z")

</div>

You formerly reported you tried `transaction_ignore_urls=/devops/alive` - did that not work as well?

Some trial and error can be useful, for example:

- `transaction_ignore_urls=*devops/alive*`
- `transaction_ignore_urls=*devops*`

etc

Also, set `log_level` to `debug` and make sure your configuration is picked up as expected.

---

<div class="post-metadata">

**Author:** ![binschlag](https://avatars.discourse-cdn.com/v4/letter/b/a4c791/32.png) [@binschlag](https://discuss.elastic.co/u/binschlag)\
**Post date:** [June 16, 2021, 8:32am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/7 "2021-06-16T08:32:12Z")

</div>

The "transaction\_ignore\_urls" is not picked up regarding DEBUG-log, i see all my parameters from config-file, but transaction\_ignore\_urls seems to be "ignored" ...

When setting this in Kibana with central configuration, I see in the logs "WARN co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Received unknown remote configuration key transaction\_ignore\_urls"

Do you have any idea why his happens?

---

<div class="post-metadata">

**Author:** ![binschlag](https://avatars.discourse-cdn.com/v4/letter/b/a4c791/32.png) [@binschlag](https://discuss.elastic.co/u/binschlag)\
**Post date:** [June 16, 2021, 10:50am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/8 "2021-06-16T10:50:41Z")

</div>

@Eyal_Koren seems like this is caused by an old version of elastic-apm-agent. I will update to newest version and then this ignore\_urls should work.

Thanks for all your feedback and help

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 7, 2021, 6:50am UTC](https://discuss.elastic.co/t/apm-filter-incoming-data/275971/9 "2021-07-07T06:50:51Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
