# Alerting on Run Failures

**URL:** <https://discuss.elastic.co/t/alerting-on-run-failures/342623>\
**Category:** Kibana\
**Tags:** elastic-stack-alerting\
**Created:** [September 8, 2023, 3:13pm UTC](https://discuss.elastic.co/t/alerting-on-run-failures/342623 "2023-09-08T15:13:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kris\_Felscher](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kris_felscher/32/125461_2.png) [@Kris\_Felscher](https://discuss.elastic.co/u/Kris_Felscher)\
**Post date:** [September 8, 2023, 3:13pm UTC](https://discuss.elastic.co/t/alerting-on-run-failures/342623/1 "2023-09-08T15:13:36Z")

</div>

We have a scheduled app that performs a nightly processing job across several different target "markets".

I'm trying to figure out how to fire off an alert when a market fails to run.

So, let's say I typically see the following events every night:

```auto
{"market" : "MarketA", "message" : "Processing Started"}
{"market" : "MarketB", "message" : "Processing Started"}
{"market" : "MarketC", "message" : "Processing Started"}

```

But then one night, I only get:

```auto
{"market" : "MarketA", "message" : "Processing Started"}
{"market" : "MarketC", "message" : "Processing Started"}

```

Is there any way to fire an alert saying "Market B Failed to Run", or will I need to create an alert for each market individually that checks for zero messages over a 24 hour period?

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [September 11, 2023, 11:46pm UTC](https://discuss.elastic.co/t/alerting-on-run-failures/342623/2 "2023-09-11T23:46:06Z")

</div>

Yes, you can do this with Watcher.

The main concept is to first query and count (with a terms aggregation) for each "market" from some prior timeframe (i.e. yesterday) and then query and count (again with a terms aggregation)for each "market" from some recent timeframe (i.e. today) and compare the two with a bucket selector. If there are "markets" that existed yesterday, but not today, then alert on that entity.

A similar example, but instead the "entity" here is an elasticsearch index:

> <https://gist.github.com/richcollier/32d291d9b960d903864113a211dbbb23>

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2023, 11:46pm UTC](https://discuss.elastic.co/t/alerting-on-run-failures/342623/3 "2023-10-09T23:46:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.