So to explain my use case. I have a general watch that alerts me if a set of keywords is found, and if they are, send me an alert. This has been working great, but I'd like to integrate something like throttling to control how often I get an email; but there is one problem I see with throttling.
Let's say for example that I want all errors "status" : "4*" (400, 401, 402, 403, etc...) to send an alert to my email. So currently I check every 5 mins for the past 6 hours of timestamps, and for example if I get a 400 and a 401 error, I'll get both error messages sent in an email. And it keeps sending until I solve it. Now let's say I throttle the watch for the 6 hours previous to prevent this. Now with throttling activated, if within that 6 hours, I happen to get another completely separate 401 from a different source that I'd like to know about, I'm guessing that the throttle won't let an email get sent off because it already reported that the watch already was sent within 6 hours. At least that is what I'm assuming.
So a) Does the throttle somehow know how many results were in the last search so that if the payload was 2 results before but now 3, that it would see the change and send off another email, even if it falls within the throttling period?
Or b) does it just see if the watch has been sent previously and not send anything for another 6 hours, regardless of the result set?
If b), is there anything I can do so that if the query retrieves the same results as the previous watch, then only send once and if the query retrieves different results form the previous watch, then send a new email with the new results?
couple of comments here, as you are touching on very different use-cases and functionalities.
First, if you want to have separate alerts for separate error codes, it might make sense to also have separate watches for this.
Second, there is no logic currently, to check for the previous results or any history. You could do this by checking the watch history manually, but this would require more work on the watcher side (i.e. a chained input, that queries the watch history as well) - which covers your b) question as well.
Instead of throttling, you might need a more complex condition in your case, as the condition when something should trigger an action or not is depending on the data being returned.
Yeah, I've been making already pretty complicated watches but I was hoping I could get throttling to take care of this one for me, but I guess I'll get to work right away on it!
Thanks for your help Alex, you've always been very helpful and responsive when I have questions and I really appreciate it
Hi,
Out of the box you probably don't get anything.
We have similar use case and we solved it by writing the alerts / sent notification in a separate index while finding the errors. So we can track if the error was already detected and we will not notify anymore. All is implemented in painless scripts of the watcher.
On the top of this we build a small visualisation in kibana to confirm messages (that record the user that confirmed based on security).
Just parsing watcher history is quite a pain and who know if format change one day.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
