Decision to throttle based on a condition

alerting

(GAUTAM SUDDAPALLI) #1

I want to throttle an action based on the result of a previously executed action.
Let's say I have 10 unique nodes for a fs utilization alert and I want to throttle based on each unique node with the met condition. For example, if Node1 was actioned and alerted the first time, it should be throttled and not actioned and alerted if it appears again within the throttle period.

Is Index Action the ideal way to do this where we store the result of execution and look it up before performing an action?

https://www.elastic.co/guide/en/x-pack/current/actions-index.html


(Guillaume Dufrenne) #2

It depend if you have 1 Watch for all nodes , or 1 Watch per nodes.
I think you should use index action to index properly the results of your watches ,so then , in your watch
you have to use multiple input :
-1st input : the query you want to do about the fs utilization
-2nd input : query the "index-results"

Use script condition to compare the result of the first input and if the "index-results" contains traces of a recently executed action and then decide what to do ..


(GAUTAM SUDDAPALLI) #3

@elastock Thanks, and sorry for being late to the party.
To your first question, yes, I have a monitoring cluster which houses metrics of 'n' clusters so I have a single watcher which alerts when any node within the 'n' cluster hit the threshold. I have 2 actions, a slack-notification which always triggers and a jira-action which should trigger only once per node per 24 hour interval.
If I have 2 indexes (monitoring and custom index) to filter on to check if the current node which has met the threshold and this node exists in the custom index before I go ahead and insert into the custom index in the action block, I would not be able to trigger my slack notification. I want both these actions to be kind of independent of each other in the same watch.
Is this something which can be done or I need to have 2 independent watches with 1 action each?


Execute actions in a Watcher independently based on result payload
(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.