APM Java Agent Ignore Error

[Ivan_Hosea]

APM Agent language and version:
1.36

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
I was wondering if there is any way to filter out specific error code and exceptions for java. For Node.js it seems to be possible with this:

How to ignore certain errors in APM?

I have a nodejs/express REST API service using the Elastic APM client, and the Elastic Cloud backend, and I'm very happy with it so far. My application regularly needs to check S3 for the existence/non-existence of certain keys in certain buckets, using the AWS S3.HeadObject method. When those keys don't exist, the S3 API returns a 404 status, which is perfectly normal behavior within my application. But the Elastic APM client reports these as Errors, which clutters up our APM dashboard and ma…

Also, for exceptions I have tried to use ignore_exceptions on elasticapm.properties based on this:

Core configuration options | APM Java Agent Reference [1.x] | Elastic

but it seems to never filter anything unless I fill in ignore_exceptions=*. I have tried *RuntimeException, *RuntimeException*, *java*, *elastic*. None of them worked.
Can anyone explain how this supposed to work?

[Jack_Shirazi]

As detailed, exception inheritance is not supported so *RuntimeException doesn't mean you get all exceptions that extend from RuntimeException. There are some examples in that link. Try setting it to the fully qualified exception class name

[Ivan_Hosea]

I tried to use *java.lang.RuntimeException now, and it still doesn't work, I used elastic's opbeans app for this: GitHub - elastic/opbeans-java: This is an implementation of the Opbeans Demo app in Java, and generating the error by accessing path /is-it-coffee-time

Could you give more specific example to ignore this error perhaps?

[Jack_Shirazi]

What is the exact error you are seeing?

[Ivan_Hosea]

Here are the error picture & its error group detail on elastic apm:

[Eyal_Koren]

The agent prints all external configurations it sees at the top of its log in INFO level.
Please verify that such lines are printed for ignore_exceptions and share them.

[Ivan_Hosea]

Here is the line:

[Eyal_Koren]

Please try setting unnest_exceptions=false and see if it makes a difference, meaning - the exception should still be reported if it is nested, which needs to be fixed, but if it will be reported as the nesting type, it will tell us why we see this.

[Ivan_Hosea]

So I tried your suggestion above, and the errors become like this at first:

So I also added:

ignore_exceptions=*org.springframework.web.util.NestedServletException

and it actually works this time. However, I can't really differ the error if it was done this way since the other error is also classified as having the same NestedServletException error. Is there any way to differ this?

[Eyal_Koren]

That was only for identifying the cause of this behavior. Now that we know why this happens, I should be able to easily fix that, so that when you set ignore_exception, it will take effect on the actual reported exception (in other words - first unnest if required and then filter).

[Ivan_Hosea]

Okay, I see, thanks for the information then, I appreciate it

[Eyal_Koren]

@Ivan_Hosea I opened a fix.
Please try it as follows:

1. remove the unnest_exceptions=false setting
2. restore your former ignore_exceptions setting
3. replace your agent with the fix snapshot
4. verify that it now works as expected

[Nekstar]

Hi @Eyal_Koren ,
I just have the same problem with unnesting and ignoring exceptions and the provided snapshot version solves it for me. Thanks.
I just have a (at least somehow) related question: Is it intended that the ignored exceptions still count against the 'Failed transaction rate'?

[Eyal_Koren]

Hi @Nekstar, welcome to the forum as well and thanks for verifying the fix

The failed transaction rate is calculated based on the number of transaction that end with error. A transaction that ends with error is such where an exception is thrown out of the transaction-related instrumented method. For example, if the agent instruments your Servlet's doGet() method to create a transaction and this method exits with an exception, then it will be considered as a failure. Unless this exception is specifically ignored, it will be reported as error as well. So, ignoring the exception type means that error events will not be sent for them, but they can still make the transaction considered as failure.

[Nekstar]

Hi, thanks for your explanation, now I got it

[Ivan_Hosea]

Hi @Eyal_Koren , I tried your fix and I can confirm that it works, thank you.

Also, at my first question I also asked about specific http error code filter for java, is this possible to do with the current module?

[Eyal_Koren]

@Ivan_Hosea Thanks for the confirmation.
The Java agent doesn't have a filtering concept as the NodeJS has.
Instead, for such things you can use ingest pipelines. In this case, I think a drop processor would be useful.

If the issue you see is related to 404 captured as error when using HttpUrlConnection, then look at the end of this issue's thread, it may be resolved with this issue's fix.

[Ivan_Hosea]

I see @Eyal_Koren , thanks for the confirmation