Setting watch to "restored" status

Elastic Stack Elasticsearch
1

Hi All,

I am wondering about a particular use case. I am using watcher to send a webhook to an external alert tracking mechanism when a watch condition is met. The external system receives the event and creates an incident ticket in our enterprise service desk. This is pretty easy to set up with watcher

Our policies dictate that when an alert ( in this case my watch ) is restored, the alerting system send a "stand down" event to the helpdesk which will close down the incident ticket. Since watches dont maintain an internal state over time, I'm wondering how to implement this.

Right now I am considering having two watches set up per condition, one for detecting "system down" events then another detecting "system ok" events. Each watch would send either the "system ok" or "system down" event to the same endpoint and update the ticket status accordingly.

To me this seems a bit kludgy and im wondering if anyone here has any thoughts on how to achieve this or has done something similar in their own implementation.

2

Hey,

this is indeed a bit clunky at the moment, we intend to improve this by being able to access data from the previous watch runs in the context in the future. What you could do now

  • Have two watches
  • Use actions with conditions, see this example

The latter one depends of course - if you always want to sent a back up event or only after somethign was down.

--Alex

3

Thanks! I think a conditional action might work for me in this case.

4

Is there a version where using conditions within an action became available? I am using elasticsearch 2.2 and receiving the error - "unknown action type [condition]"

my action looks like this

"actions": {
"notify-system-down": {
"condition": {
"compare": { "ctx.payload.aggregations.average-response-time.value" : { "gt" : 100000 }}
},
"webhook": {
"method": "POST",
"host": "",
"port": 4019,
"path": "/eventlistener/createEvent",
"headers": {
"Content-Type": "text/xml"
},
"body": ""
}
}
}

5

Hey,

this was introduced in Elasticsearch 5.1. See the respective release notes

--Alex

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.