Watcher - Sending x number of emails per hit

Elastic Stack Elasticsearch
1

Hi,

I'm trying to setup a watcher so that based on the number on hits in my query send out that number of emails.
It seems to me that this can't be done by simply looping around the email action so I have an idea but need help on the implementation.

My idea is to grab the top hit sorted by time (ascending), save that hit using an index action, and then send an email out for that hit. Also in the query will be a script query that will filter out hits that have already been alerted. ie. Comparing with doc created by index action on certain fields

What I am missing so far is getting the index action to create a doc based on my hit and then the compare script query. (I have removed my email address but i can confirm that that is working fine)

My first priority is for my hit to be created as a doc. Example below.

{
    "trigger": {
        "schedule": {
            "interval": "1m"
        }
    },
    "input": {
        "search": {
            "request": {
                "search_type": "query_then_fetch",
                "indices": [
                    "metricbeat*"
                ],
                "types": [],
                "body": {
                    "from": 0,
                    "size": 1,
                    "query": {
                        "bool": {
                            "filter": {
                                "bool": {
                                    "must": [
                                        {
                                            "range": {
                                                "@timestamp": {
                                                    "gte": "{{ctx.trigger.scheduled_time}}||-60m",
                                                    "lte": "{{ctx.trigger.scheduled_time}}",
                                                    "format": "strict_date_optional_time||epoch_millis"
                                                }
                                            }
                                        },
                                        {
                                            "range": {
                                                "system.cpu.total.pct": {
                                                    "gt": "0.2"
                                                }
                                            }
                                        }
                                    ]
                                }
                            }
                        }
                    },
                    "sort": {
                        "@timestamp": {
                            "order": "asc"
                        }
                    }
                }
            }
        }
    },
    "condition": {
        "compare": {
            "ctx.payload.hits.total": {
                "gte": 1
            }
        }
    },
    "actions": {
        "index_payload": {
            "index": {
                "index": "metricbeat-watcher",
                "doc_type": "doc"
            }
        },
        "send_email": {
            "email": {
                "profile": "standard",
                "to": [
                    ""
                ],
                "body": {
                    "text": "There are {{ctx.payload.hits.total}} documents in your index. Threshold is 1. {{ctx.payload.hits.hits.0._source.system.cpu.total.pct}}"
                }
            }
        }
    }
}

When I try to search for any documents on my index metricbeat-watcher I get zero hits so i'm thinking that some type of transform is needed in the index action? (i get over 1000 hits for hits total so no problem with that)

If anyone has any suggestions or alternative solutions that would be much appreciated.

Regards, Patrick

2

Hey,

this is currently not possible with watcher, an action can only be executed once. A valid workaround would be to maybe send your data to logstash, process it using the http input and then use the email output plugin to send an email?

--Alex

3

Thanks Alex, i'll take a look at your suggestion! In my current example though I am only trying to execute each action once (index and email). Just wondering why my index action is not currently working or if my syntax is incorrect (is a transform required in an index action?).
Snippet from above:

"index_payload": {
    "index": {
        "index": "metricbeat-watcher",
        "doc_type": "doc"
    }
}

Regards, Patrick

4

please include the output of the Execute Watch API here for further debugging

5

Hi Alex,

Of course!
The output was too big to post here so I have shared the output at the link below.
Watch Output

Regards, Patrick

6

hey Patrick,

that looks like a successful run.

A new document has been created as metricbeat-watcher/doc/c-7h8mIB1iKSZXGXi6FM and the email seems to have been sent successful as well.

You can see it in watch_record.result.actions.0.index.response.

Hope this helps.

--Alex

7

Thanks Alex, you have been a great help! I think the issue was that I was just using the Discover page on Kibana and couldn't see results from there. I can see there have been multiple docs created using a GET request from the console.

Regards, Patrick

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.