Please excuse my newbie question, I've been trying to figure it out using the docs by myself, however, I failed.
I'd like to do this: I have an index which contains alerts. I'm trying to configure a watch that is triggered every 24 hours. It looks for certain alerts over the last 24h and normally, there are several search results for this search within 24 hours. This means we get quite a large number of hits with one search. I can actually see the payload, it looks correct. I'd like to generate an email for each of the results within the payload, however, I have no Idea how to do it. Furthermore, do you have any advice how to access each result in the payload (maybe with something like a "for" loop) and within the result access certain fields?
I really like the watcher docs, however, I think they could be improved by including some more examples because I realized that there're certain scenarios that are hard to figure out using only the docs without any examples. Watcher is really complex and powerful and the learning curve is quite steep.
I'm afraid this isn't easily do-able today. The indexing action is currently the only one that allows multiple action executions based on the results. This is definitely a feature we'd like to add!
In your specific case, there might be some workarounds, though they might add a fair amount of complexity.
First, what process is creating the index full of of alerts? Could you convert whatever generates the alerts over to using Watcher, so you could just send the alert information as soon as it happens with Watcher?
If that's not suitable, it might be possible to use chained inputs to retrieve information about the last alert you sent notification about, then use that information to run a query that retrieves the next 1 item, and send the notification, and update the "last sent" value using an index action to update a doc that holds this info.
I agree - this is far more complicated than it should be.. which is why we have "run a single action multiple times based on an array in the results" feature on the roadmap!
Regarding your question about the nature of the process that causes the alerts: we try to process IPS alerts. The crux is that we would like to do a periodic 24h processing but the IPS may throw the same alert over and over again for the same host and the same threat. However, we only want a single email action to take place for the same type of alert within -- say -- 24h because it's sufficient to alert an admin once about a certain threat and not each time the threat takes place.
I've been thinking about chained inputs and using a separate index to store information about alerts that have already been handled within the last 24h. However, if I understand the watcher correctly, this calls for two separate watches: one which has an index action for the "bookkeeping" and one that actually sends the email. This strikes me as being really complicated. The more I think about it the more I'd appreciate the following two features: the "run a single action multiple times based on an array in the results" feature that you mentioned as well as a "chained actions" feature which could do the "bookkeeping" with the index action and the email action within a single watch. I hope that this makes sense.
Thanks for the additional background! If you were generating the alerts from Watcher, with one watcher per alert type, as soon as the IPS finds them, rather than batching them up, you could use the "Throttle period" setting of Watcher to ensure that you only send one email notification for a given time period. If that sort of thing isn't practical, no worries
You can already perform multiple actions in a single watch - the actions are an array. Each action can also have it's own transform stage, so you can appropriately format things for templating into the action. See the second watch example in this section for an example of multiple actions for a single watch. They can even have different throttle periods, etc.
Oh! Thanks for pointing this out, I completely missed it when reading the docs. This is good news.
My guess is that putting each alert type of an IPS into a separate watch is not practical because with each threat signature update, new threats can emerge and that makes it hard to keep up with adding new watches. We'll ponder about this a little more and hopefully find a good solution because we really like ELK and we think that watcher will help us streamline some processes.
just to add here to one part of your first question, how to loop through results, i.e. inside of your email body... (this is on top of my head, so maybe I am missing a field)
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
