# Multi Metric Watcher won't trigger Action even though condition is met **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-alerting, painless\ **Created:** [June 21, 2021, 4:19am UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474 "2021-06-21T04:19:25Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Rohan-boogeyman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohan-boogeyman/32/81377_2.png) [@Rohan-boogeyman](https://discuss.elastic.co/u/Rohan-boogeyman)\ **Post date:** [June 21, 2021, 4:19am UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/1 "2021-06-21T04:19:26Z")
Hello guys, Hope you are all doing great - need some assistance on this issue i am experiencing where when i try to Simulate this watcher - it won't trigger the "action" it just leaves action blank as if conditions weren't met - when they are met. ```auto { "watch_id": "_inlined_", "node": "O5vw_C5lTvq_VZ8EJwV-4g", "state": "execution_not_needed", "user": "User_ID", "status": { "state": { "active": true, "timestamp": "2021-06-21T04:11:57.130Z" }, "last_checked": "2021-06-21T04:11:57.134Z", "actions": { "Watcher-webhook": { "ack": { "timestamp": "2021-06-21T04:11:57.130Z", "state": "awaits_successful_execution" } } }, "execution_state": "execution_not_needed", "version": -1 }, "trigger_event": { "type": "manual", "triggered_time": "2021-06-21T04:11:57.131Z", "manual": { "schedule": { "scheduled_time": "2021-06-21T04:11:57.131Z" } } }, "input": { "search": { "request": { "search_type": "query_then_fetch", "indices": [ ".ml-anomalies-*" ], "rest_total_hits_as_int": true, "body": { "size": 0, "query": { "bool": { "filter": [ { "term": { "job_id": "all-errors" } }, { "range": { "timestamp": { "gte": "now-30m" } } }, { "terms": { "result_type": [ "bucket", "record", "influencer" ] } } ] } }, "aggs": { "bucket_results": { "filter": { "range": { "anomaly_score": { "gte": 0 } } }, "aggs": { "top_bucket_hits": { "top_hits": { "sort": [ { "anomaly_score": { "order": "desc" } } ], "_source": { "includes": [ "job_id", "result_type", "timestamp", "anomaly_score", "is_interim" ] }, "size": 1, "script_fields": { "start": { "script": { "lang": "painless", "source": "LocalDateTime.ofEpochSecond((doc[\"timestamp\"].value.getMillis()-((doc[\"bucket_span\"].value * 1000)\n * params.padding)) / 1000, 0, ZoneOffset.UTC).toString()+\":00.000Z\"", "params": { "padding": 10 } } }, "end": { "script": { "lang": "painless", "source": "LocalDateTime.ofEpochSecond((doc[\"timestamp\"].value.getMillis()+((doc[\"bucket_span\"].value * 1000)\n * params.padding)) / 1000, 0, ZoneOffset.UTC).toString()+\":00.000Z\"", "params": { "padding": 10 } } }, "timestamp_epoch": { "script": { "lang": "painless", "source": "doc[\"timestamp\"].value.getMillis()/1000" } }, "timestamp_iso8601": { "script": { "lang": "painless", "source": "doc[\"timestamp\"].value" } }, "score": { "script": { "lang": "painless", "source": "Math.round(doc[\"anomaly_score\"].value)" } } } } } } }, "influencer_results": { "filter": { "range": { "influencer_score": { "gte": 3 } } }, "aggs": { "top_influencer_hits": { "top_hits": { "sort": [ { "influencer_score": { "order": "desc" } } ], "_source": { "includes": [ "result_type", "timestamp", "influencer_field_name", "influencer_field_value", "influencer_score", "isInterim" ] }, "size": 3, "script_fields": { "score": { "script": { "lang": "painless", "source": "Math.round(doc[\"influencer_score\"].value)" } } } } } } }, "record_results": { "filter": { "range": { "record_score": { "gte": 3 } } }, "aggs": { "top_record_hits": { "top_hits": { "sort": [ { "record_score": { "order": "desc" } } ], "_source": { "includes": [ "result_type", "timestamp", "record_score", "is_interim", "function", "field_name", "by_field_value", "over_field_value", "partition_field_value" ] }, "size": 3, "script_fields": { "score": { "script": { "lang": "painless", "source": "Math.round(doc[\"record_score\"].value)" } } } } } } } } } } } }, ``` ---
**Author:** ![Rohan-boogeyman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohan-boogeyman/32/81377_2.png) [@Rohan-boogeyman](https://discuss.elastic.co/u/Rohan-boogeyman)\ **Post date:** [June 21, 2021, 4:20am UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/2 "2021-06-21T04:20:14Z")
```auto "condition": { "compare": { "ctx.payload.aggregations.bucket_results.doc_count": { "gt": 0 } } }, "metadata": { "xpack": { "type": "json" } }, "result": { "execution_time": "2021-06-21T04:11:57.134Z", "execution_duration": 17, "input": { "type": "search", "status": "success", "payload": { "_shards": { "total": 2, "failed": 0, "successful": 2, "skipped": 0 }, "hits": { "hits": [], "total": 0, "max_score": null }, "took": 14, "timed_out": false, "aggregations": { "influencer_results": { "doc_count": 0, "top_influencer_hits": { "hits": { "hits": [], "total": 0, "max_score": null } } }, "record_results": { "doc_count": 0, "top_record_hits": { "hits": { "hits": [], "total": 0, "max_score": null } } }, "bucket_results": { "top_bucket_hits": { "hits": { "hits": [], "total": 0, "max_score": null } }, "doc_count": 0 } } }, "search": { "request": { "search_type": "query_then_fetch", "indices": [ ".ml-anomalies-*" ], "rest_total_hits_as_int": true, "body": { "size": 0, "query": { "bool": { "filter": [ { "term": { "job_id": "all-errors" } }, { "range": { "timestamp": { "gte": "now-30m" } } }, { "terms": { "result_type": [ "bucket", "record", "influencer" ] } } ] } }, "aggs": { "bucket_results": { "filter": { "range": { "anomaly_score": { "gte": 0 } } }, "aggs": { "top_bucket_hits": { "top_hits": { "sort": [ { "anomaly_score": { "order": "desc" } } ], "_source": { "includes": [ "job_id", "result_type", "timestamp", "anomaly_score", "is_interim" ] }, "size": 1, "script_fields": { "start": { "script": { "lang": "painless", "source": "LocalDateTime.ofEpochSecond((doc[\"timestamp\"].value.getMillis()-((doc[\"bucket_span\"].value * 1000)\n * params.padding)) / 1000, 0, ZoneOffset.UTC).toString()+\":00.000Z\"", "params": { "padding": 10 } } }, "end": { "script": { "lang": "painless", "source": "LocalDateTime.ofEpochSecond((doc[\"timestamp\"].value.getMillis()+((doc[\"bucket_span\"].value * 1000)\n * params.padding)) / 1000, 0, ZoneOffset.UTC).toString()+\":00.000Z\"", "params": { "padding": 10 } } }, "timestamp_epoch": { "script": { "lang": "painless", "source": "doc[\"timestamp\"].value.getMillis()/1000" } }, "timestamp_iso8601": { "script": { "lang": "painless", "source": "doc[\"timestamp\"].value" } }, "score": { "script": { "lang": "painless", "source": "Math.round(doc[\"anomaly_score\"].value)" } } } } } } }, "influencer_results": { "filter": { "range": { "influencer_score": { "gte": 3 } } }, "aggs": { "top_influencer_hits": { "top_hits": { "sort": [ { "influencer_score": { "order": "desc" } } ], "_source": { "includes": [ "result_type", "timestamp", "influencer_field_name", "influencer_field_value", "influencer_score", "isInterim" ] }, "size": 3, "script_fields": { "score": { "script": { "lang": "painless", "source": "Math.round(doc[\"influencer_score\"].value)" } } } } } } }, "record_results": { "filter": { "range": { "record_score": { "gte": 3 } } }, "aggs": { "top_record_hits": { "top_hits": { "sort": [ { "record_score": { "order": "desc" } } ], "_source": { "includes": [ "result_type", "timestamp", "record_score", "is_interim", "function", "field_name", "by_field_value", "over_field_value", "partition_field_value" ] }, "size": 3, "script_fields": { "score": { "script": { "lang": "painless", "source": "Math.round(doc[\"record_score\"].value)" } } } } } } } } } } } }, "condition": { "type": "compare", "status": "success", "met": false, "compare": { "resolved_values": { "ctx.payload.aggregations.bucket_results.doc_count": 0 } } }, "actions": [] }, "messages": [] } ``` ---
**Author:** ![Rohan-boogeyman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohan-boogeyman/32/81377_2.png) [@Rohan-boogeyman](https://discuss.elastic.co/u/Rohan-boogeyman)\ **Post date:** [June 21, 2021, 4:20am UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/3 "2021-06-21T04:20:48Z")
Apologies had to divide my code into parts cause it wont take whole content due to restriction of letters ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [June 21, 2021, 8:31am UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/4 "2021-06-21T08:31:14Z")
> [@Rohan-boogeyman](#): > > ```auto > "condition": { > "type": "compare", > "status": "success", > "met": false, > "compare": { > "resolved_values": { > "ctx.payload.aggregations.bucket_results.doc_count": 0 > } > } > } > > ``` The interesting part of the response is quoted above. Your search request does not contain any hits, thus the condition is not met. ---
**Author:** ![Rohan-boogeyman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohan-boogeyman/32/81377_2.png) [@Rohan-boogeyman](https://discuss.elastic.co/u/Rohan-boogeyman)\ **Post date:** [June 21, 2021, 10:20am UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/5 "2021-06-21T10:20:39Z")
thanks @spinscale - if you look at my watcher then basically my anomaly score is set to 0 - and in such condition shouldn't we be expecting a confirmed hit? ```auto { "trigger": { "schedule": { "interval": "119s" } }, "input": { "search": { "request": { "search_type": "query_then_fetch", "indices": [ ".ml-anomalies-*" ], "rest_total_hits_as_int": true, "body": { "size": 0, "query": { "bool": { "filter": [ { "term": { "job_id": "all-errors" } }, { "range": { "timestamp": { "gte": "now-30m" } } }, { "terms": { "result_type": [ "bucket", "record", "influencer" ] } } ] } }, "aggs": { "bucket_results": { "filter": { "range": { "anomaly_score": { "gte": 0 } } }, "aggs": { "top_bucket_hits": { "top_hits": { "sort": [ { "anomaly_score": { "order": "desc" } } ], "_source": { "includes": [ "job_id", "result_type", "timestamp", "anomaly_score", "is_interim" ] }, "size": 1, "script_fields": { "start": { "script": { "lang": "painless", "source": "LocalDateTime.ofEpochSecond((doc[\"timestamp\"].value.getMillis()-((doc[\"bucket_span\"].value * 1000)\n * params.padding)) / 1000, 0, ZoneOffset.UTC).toString()+\":00.000Z\"", "params": { "padding": 10 } } }, "end": { "script": { "lang": "painless", "source": "LocalDateTime.ofEpochSecond((doc[\"timestamp\"].value.getMillis()+((doc[\"bucket_span\"].value * 1000)\n * params.padding)) / 1000, 0, ZoneOffset.UTC).toString()+\":00.000Z\"", "params": { "padding": 10 } } }, "timestamp_epoch": { "script": { "lang": "painless", "source": """doc["timestamp"].value.getMillis()/1000""" } }, "timestamp_iso8601": { "script": { "lang": "painless", "source": """doc["timestamp"].value""" } }, "score": { "script": { "lang": "painless", "source": """Math.round(doc["anomaly_score"].value)""" } } } } } } }, "influencer_results": { "filter": { "range": { "influencer_score": { "gte": 3 } } }, "aggs": { "top_influencer_hits": { "top_hits": { "sort": [ { "influencer_score": { "order": "desc" } } ], "_source": { "includes": [ "result_type", "timestamp", "influencer_field_name", "influencer_field_value", "influencer_score", "isInterim" ] }, "size": 3, "script_fields": { "score": { "script": { "lang": "painless", "source": """Math.round(doc["influencer_score"].value)""" } } } } } } }, "record_results": { "filter": { "range": { "record_score": { "gte": 3 } } }, "aggs": { "top_record_hits": { "top_hits": { "sort": [ { "record_score": { "order": "desc" } } ], "_source": { "includes": [ "result_type", "timestamp", "record_score", "is_interim", "function", "field_name", "by_field_value", "over_field_value", "partition_field_value" ] }, "size": 3, "script_fields": { "score": { "script": { "lang": "painless", "source": """Math.round(doc["record_score"].value)""" } } } } } } } } } } } }, "condition": { "compare": { "ctx.payload.aggregations.bucket_results.doc_count": { "gt": 0 } } }, "actions": { "Watcher-webhook": { "throttle_period_in_millis": 30000, "webhook": { "scheme": "http", "host": "xxxxx", "port": xxx, "method": "post", "path": ":/alert/{{ctx.watch_id}}", "params": {}, "headers": { "Content-Type": "application/json" }, "body": """{ "signature": "[{{ctx.execution_time}}]", "source": "xxxx","environment": "xxxx", "source_id": "xxxx [{{ctx.payload.aggregations.bucket_results.top_bucket_hits.hits.hits.0._source.job_id}}]", "ApplicationCode": "xxx", "external_id":"xxx", "agent":"xxxx", "agent_location":"xxx", "severity": "CRITICAL", "moogsoft_type": "xxx", "manager": "xxxx", "class": "application","createTicket": "1" , "description": "Alert for job [{{ctx.payload.aggregations.bucket_results.top_bucket_hits.hits.hits.0._source.job_id}}] at [{{ctx.payload.aggregations.bucket_results.top_bucket_hits.hits.hits.0.fields.timestamp_iso8601.0}}] score [{{ctx.payload.aggregations.bucket_results.top_bucket_hits.hits.hits.0.fields.score.0}}]" }""" } } } } ``` ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [June 21, 2021, 1:04pm UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/6 "2021-06-21T13:04:25Z")
If you have security enabled, than the user, who you are storing the watch as, needs to have privileges to read that index. Is that given? You can check by trying out that query in the dev-tools console. ---
**Author:** ![Rohan-boogeyman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohan-boogeyman/32/81377_2.png) [@Rohan-boogeyman](https://discuss.elastic.co/u/Rohan-boogeyman)\ **Post date:** [June 21, 2021, 3:13pm UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/7 "2021-06-21T15:13:53Z")
Hi @spinscale this watcher was created as a part of ML - and yes user does have access to the Index - I tried executing using DEV Tools and below is the output i got - knowing the watcher is already created - i simply tried to post that same watcher again { "\_id" : "all-errors", "\_version" : 38032, "\_seq\_no" : 5235590, "\_primary\_term" : 2, "created" : false } ---
**Author:** ![Rohan-boogeyman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rohan-boogeyman/32/81377_2.png) [@Rohan-boogeyman](https://discuss.elastic.co/u/Rohan-boogeyman)\ **Post date:** [June 22, 2021, 12:05am UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/8 "2021-06-22T00:05:30Z")
Just to Add - so i created a Single Metric ML and executed watcher - it seems to work fine. This is happening with Multi Metric ML mostly - Watcher is created as a Part of ML - and at the time of "Simulation" i am making change to the parameter anomoly\_score from 75 to 0 - Now this works perfectly fine if its a Single Metric watcher - How can be Simulate a Multi Metric Watcher by passing manual values? because it seems there's more than just "anomoly\_score" when it comes to Multi Metric Watcher ```auto "aggs": { "bucket_results": { "filter": { "range": { "anomaly_score": { "gte": 0 } } }, ``` ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [June 22, 2021, 2:14pm UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/9 "2021-06-22T14:14:06Z")
To get a better understanding of ML results information, and how it relates to alerting may I suggest: - This blog: [Machine Learning Anomaly Scoring and Elasticsearch - How it Works | Elastic Blog](https://www.elastic.co/blog/machine-learning-anomaly-scoring-elasticsearch-how-it-works) - This blog: [Elastic Machine Learning Anomaly Scoring with Partition Normalization and Multi-Bucket Analysis | Elastic Blog](https://www.elastic.co/blog/changes-to-elastic-machine-learning-anomaly-scoring-in-6-5) - Chapters 5 and 6 of this book: [https://www.amazon.com/gp/product/1801070032](https://www.amazon.com/gp/product/1801070032) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 20, 2021, 2:15pm UTC](https://discuss.elastic.co/t/multi-metric-watcher-wont-trigger-action-even-though-condition-is-met/276474/10 "2021-07-20T14:15:03Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.