Question about multiple conditions, actions and throttling

Elastic Stack Elasticsearch
1

Hi all,

I was wondering if it's possible to have multiple conditions in one watch and whether its possible to limit throttling to 1 alert per day/week/month.

Conditions:
I'm checking for data usage. If usage is over a certain value I want to generate an email alert. However I want to set alert at 3 different intervals, lets say 10GB, 20GB and 50GB.

Is this possible?

Throttling:
Taking the above example, I want generate only one alert per value per month as the usage counter will reset every month so there is no point in generating more than one alert (support won't like that either).

Is this possible?

Actions:
Again taking the above case, for each alert I want to send an email but I might want to have three different emails ranging from be careful to better turn off you device until support contact you.

Is this possible?

My watch

{
  "trigger": {
    "schedule": {
      "interval": "30m"
    }
  },
 "input": {
    "search": {
      "request": {
        "indices": [ "netflow-*" ],
        "types": "netflow",
  "body": {
    "query": {
        "bool": {
            "must" : {
                "match": {
                    "host": "1.1.1.1"
                }
            },
            "filter": {
                "range" : {
                    "@timestamp": {
                      "gte": "now-1M/M",
                      "lte": "now"
                    }
                }
            }
        }
    },
       "aggs" : {
      "total" : { "sum" : { "field" : "netflow.in_bytes"} }
   },
    "size": 0
  }
  }
}
},
  "condition": {
    "compare": {
      "ctx.payload.aggregations.total.value": {
        "gte": 1073741824
      }
    }
  },
  "actions": {
    "my-logging-action": {
      "logging": {
        "text": "Limit exceeded."
      }
    }
  }
}
2

Hey,

so there is time based throttling in watcher, allowing you to throttle for a certain amount of time (or unless the condition becomes false again).

In this example you could have a single watch, which has three logging outputs and each of those outputs have

  • a condition (greather than 10, 20 or 50 GB)
  • a time based throttle

--Alex

3

@spinscale does throttling support time based logic to set the trigger to the 1st day of the next month? The docs only mention specifying a single value such as 15m.

I can't find anything with regards stopping alerts until a value becomes false again either outside of the watch API? But if I use the API that would require something like running a script on the server each month to reset the condition? Is that supported on the standard cloud subscription?

The condition requires a separate "condition": { for each value? I assume its not as simple as "gte" 10GB OR 20GB OR 50GB?

4

Hey,

that logic regarding the first of the month, needs to be implemented yourself, the thottling only supports time spans.

How about this: In your watch, you are going to index each user, that is violating the traffic rules into an own index. Now when you execute a watch, you can compare that list of users with the list of users returned by the query and dismiss them, if they already received a notification.

Then add a second watch, that runs on the first of every month and deletes the index with traffic violators.

--Alex

5

Yes, the latter sounds like a more viable solution given my lack of skills. I did think about that a little bit earlier this week when I found the docs page showing an example of writing to an index but I haven't managed to find that page again...

I'll let this sink in over the weekend and give it a go next week.

Thanks.

6

guess you are searching for the index action