Elasticsearch APM Missing Alerts

[Shameek_Agarwal]

hi all, really liking apm so far!
so basically, i am doing via opentelemetry (logs + traces only for now)

now, i want to alert for e.g. by email whenever any error occurs
e.g. if traces / logs from opentelemetry observe any error, send an alert
its not like i want to alert only on some breach
this is just because of the nature of apps i am working with

while i did get the email functionality to work
what i observed was that i am missing out on some errors / emails
is there something obvious i am missing ?
also, are there some suggestions for better configuring this based on my requirements?

Screenshot of my Alert Configuration

so, make note of -

1. error count threshold - above 0 errors
2. for last 5 minutes
3. poll every 5 minutes

[Kate_Patticha]

Hi @Shameek_Agarwal

@Shameek_Agarwal

what i observed was that i am missing out on some errors / emails

Can you please elaborate more on that?

[Shameek_Agarwal]

Thaks for getting back @Kate_Patticha !

basically, i know how to create an error in my applications
there is a get endpoint, and if i pass it an id of -1, it results in an error

i would expect this - all errors that happen in a span of 5 minutes and show up as a single alert (maybe per service?)

i call that endpoint in the 1st and e.g. 6th minute, i dont get an email twice, it is flaky

so, i was interested in knowing how (and if) i can improve the reliability for this alerting mechanism, based on my requirements and the configuration i put above

also, the notify dropdown, is it better being "on status changes" or "on check interval"? cannot find the documentation for it / unable to properly wrap my head around it...

[Kate_Patticha]

I understand; it seems to be the same error or grouped under the same error grouping key, right? I think that might explain why you're not receiving additional emails while selected on status change.

When utilizing the 'on check interval' setting, notifications will be triggered if the specified condition is met at the scheduled interval. In that case, you should receive an additional email.