Mute certain APM errors from watcher

Kibana version : 7.6.2
Elasticsearch version : 7.6.2
APM Server version :7.6.2
APM Agent language and version : Node.JS 3.5.0
Original install method : Elastic Cloud

Is there anything special in your setup? No

Issue
We use Watcher to email us every X minutes any APM errors that have occured in X time. Sometimes there are errors that we want to see in Kibana but don't want Watcher to notify us of. Is there a simple way to exclude certain errors based on a regex style string matching (i.e. mute the error)?

Use Case
We have some errors that we are dealing with or are not critical, so once we know about the error we don't want more emails about this error as by this point we have created a task for this issue in our management software. Currently, some errors are cluttering our inboxes!

Possible Resolutions
There is a potential resolution being implement by Elastic (https://github.com/elastic/kibana/pull/50128), we have waited a while for this now but not much seems to have occurred on this for a while now and we now need a temporary solution.

We are trying to avoid a cumbersome solution and keep it clean and quick to update with new strings, hence, the first two options are not ideal):

  • editing the Watch to exclude matching strings
  • adding in logic to our services to add an extra "mute" field (which is used to exclude the log from the Watch)
  • (not sure how possible this is) create a separate index which stores regex style strings to ignore and somehow use Watcher to excluding any error logs matching any of the strings from this new extra index
  • ?

This may not be the correct category for this post but there was no "Watcher" category - apologies if it is wrong

Thanks very much! Matthew

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.