Optimize Action in Alerts to trigger only once for a detected anomaly

alerting

(Prince) #1

Hi Experts,

I need to optimize the alerts as triggered in scheduled interval in which the action is to be executed only once for detected anomalies and the same alert should not execute the action until the anomaly is resolved and execute the action again if the same anomaly is detected again.

can you help me in creating the alert as above statement?

Thanks in advance

  • Elango

(Andrés Pérez) #2

You can set a throttle_period long enough to ensure that the anomalies are resolved.

If the current anomaly (i.e. what makes the condition come true) is resolved within the throttle period:

  • the next watch execution won't have it's condition met so...
  • no actions will be executed and
  • the alert will be "rearmed", ready for the next anomaly.

A similar question has been asked here recently, you can get more information there: How to stop sending duplicate Slack notifications for the same error?


(Alexander Reelsen) #3

throttle period is one possible solution to this - but it might be limited.

If you really need to check for previous executions, a chain input, where the first input consists of querying the last three watch history entries for this watch might be helpful. This way you could detect also flapping of your check, if you need to.

Hope this helps!

--Alex


(Prince) #4

Thank you @andres-perez and @spinscale for the suggestions I am try to work this out.

Will update once I am done

:slight_smile: Elango MAS


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.