Watcher - Execution history error state

Hello,

I have created a watcher to be alerted when the price of an item goes up too much. It is a piece of information that I receive from enterprise search and that I have already implemented in a dashboard.


When I let the watcher work, each time it is triggered, its status is error and I do not receive any emails. It is important to specify that the mail test works !

Here is the output I receive:

{
    "watch_id": "8f7b98b3-12ff-4077-b34e-5d20701c137d",
    "node": "fhCdip7tRgmP7bb7UlYYDQ",
    "state": "failed",
    "user": "3197049516",
    "status": {
      "state": {
        "active": true,
        "timestamp": "2022-02-14T11:50:38.237Z"
      },
      "actions": {
        "email_1": {
          "ack": {
            "timestamp": "2022-02-14T11:50:38.237Z",
            "state": "awaits_successful_execution"
          }
        }
      },
      "execution_state": "failed",
      "version": -1
    },
    "trigger_event": {
      "type": "schedule",
      "triggered_time": "2022-02-14T12:10:38.499Z",
      "schedule": {
        "scheduled_time": "2022-02-14T12:10:38.242Z"
      }
    },
    "input": {
      "search": {
        "request": {
          "search_type": "query_then_fetch",
          "indices": [
            ".ent-search-engine-documents-*"
          ],
          "rest_total_hits_as_int": true,
          "body": {
            "size": 0,
            "query": {
              "bool": {
                "filter": {
                  "range": {
                    "date.date": {
                      "gte": "{{ctx.trigger.scheduled_time}}||-1d",
                      "lte": "{{ctx.trigger.scheduled_time}}",
                      "format": "strict_date_optional_time||epoch_millis"
                    }
                  }
                }
              }
            },
            "aggs": {
              "metricAgg": {
                "avg": {
                  "field": "billingappv3.float"
                }
              }
            }
          }
        }
      }
    },
    "condition": {
      "script": {
        "source": "if (ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;",
        "lang": "painless",
        "params": {
          "threshold": 1.2
        }
      }
    },
    "metadata": {
      "name": "BillingAppV3",
      "watcherui": {
        "trigger_interval_unit": "m",
        "agg_type": "avg",
        "time_field": "date.date",
        "trigger_interval_size": 10,
        "term_size": 5,
        "time_window_unit": "d",
        "threshold_comparator": ">",
        "index": [
          ".ent-search-engine-documents-*"
        ],
        "time_window_size": 1,
        "threshold": 1.2,
        "agg_field": "billingappv3.float"
      },
      "xpack": {
        "type": "threshold"
      }
    },
    "result": {
      "execution_time": "2022-02-14T12:10:38.499Z",
      "execution_duration": 3,
      "input": {
        "type": "search",
        "status": "success",
        "payload": {
          "_shards": {
            "total": 2,
            "failed": 0,
            "successful": 2,
            "skipped": 0
          },
          "hits": {
            "hits": [],
            "total": 0,
            "max_score": null
          },
          "took": 1,
          "timed_out": false,
          "aggregations": {
            "metricAgg": {
              "value": null
            }
          }
        },
        "search": {
          "request": {
            "search_type": "query_then_fetch",
            "indices": [
              ".ent-search-engine-documents-*"
            ],
            "rest_total_hits_as_int": true,
            "body": {
              "size": 0,
              "query": {
                "bool": {
                  "filter": {
                    "range": {
                      "date.date": {
                        "gte": "2022-02-14T12:10:38.242Z||-1d",
                        "lte": "2022-02-14T12:10:38.242Z",
                        "format": "strict_date_optional_time||epoch_millis"
                      }
                    }
                  }
                }
              },
              "aggs": {
                "metricAgg": {
                  "avg": {
                    "field": "billingappv3.float"
                  }
                }
              }
            }
          }
        }
      },
      "actions": []
    },
    "exception": {
      "type": "script_exception",
      "reason": "runtime error",
      "script_stack": [
        "if (ctx.payload.aggregations.metricAgg.value > params.threshold) { ",
        "                                                     ^---- HERE"
      ],
      "script": "if (ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;",
      "lang": "painless",
      "position": {
        "offset": 53,
        "start": 0,
        "end": 67
      },
      "caused_by": {
        "type": "null_pointer_exception",
        "reason": "Cannot invoke \"Object.getClass()\" because \"leftObject\" is null",
        "stack_trace": "java.lang.NullPointerException: Cannot invoke \"Object.getClass()\" because \"leftObject\" is null\n\tat org.elasticsearch.painless.DefBootstrap$MIC.checkBoth(DefBootstrap.java:432)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(if (ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;:54)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:65)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:60)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:543)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:340)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$5(ExecutionService.java:441)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:670)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:718)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat java.base/java.lang.Thread.run(Thread.java:833)\n"
      },
      "stack_trace": "ScriptException[runtime error]; nested: NullPointerException[Cannot invoke \"Object.getClass()\" because \"leftObject\" is null];\n\tat org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:85)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(if (ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;:1)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:65)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:60)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:543)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:340)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$5(ExecutionService.java:441)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:670)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:718)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat java.base/java.lang.Thread.run(Thread.java:833)\nCaused by: java.lang.NullPointerException: Cannot invoke \"Object.getClass()\" because \"leftObject\" is null\n\tat org.elasticsearch.painless.DefBootstrap$MIC.checkBoth(DefBootstrap.java:432)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(if (ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;:54)\n\t... 10 more\n"
    }
  }

Can you please help me?

Hey,

your script tries to access a part of the search response via

"if (ctx.payload.aggregations.metricAgg.value > params.threshold) { ",

you can not configure parameter with a watcher script. Try replacing with a hard coded value.

--Alex

Thank you for your answer.
I don't know if I understood your answer very well but I'm not using any custom script. I'm using the watcher page in Kibana > Stack Management on Elastic Cloud.
So, i don't use any parameter.

just to be sure I understand: You used the Stack Management UI and all you did was specifying the threshold or you wrote the whole alerting watch including the condition yourself?

Also, which version is this?

Well, I defined the threshold and the condition while going through the stack management interface. I didn't touch any script by myself, I just used the available tools as shown on the first screenshot.

My deployment uses version 7.16.3.

Hey,

so I figured out the issue... that watch assumes that your query matches documents. If that is not the case this error occurs...

changing the condition to

if (ctx.payload.aggregations.metricAgg.value != null && ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;

fixes this, but of course that is a bug and should be fixed within the Alerting UI.

I will open an issue! Thanks for sticking with me and reporting this.

--Alex

1 Like

Opened Threshold based alert throws exception when no documents are returned · Issue #126074 · elastic/kibana · GitHub

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.