ML watcher error

Elastic Stack Kibana
1

Hiya,

I created a watcher for alerting when anomalies happens in a machine learning job, I also changed the action to alert through slack instead of mail. But when trying to simulate I get an error, I am still a noobie about this so I would really aprecciate your help.
Version 7.2.0

The output when I simulate it:

 "actions": [
      {
        "id": "log",
        "type": "logging",
        "status": "failure",
        "error": {
          "root_cause": [
            {
              "type": "general_script_exception",
              "reason": "Error running com.github.mustachejava.codes.DefaultMustache@a569020"
            }
          ],
          "type": "general_script_exception",
          "reason": "Error running com.github.mustachejava.codes.DefaultMustache@a569020",
          "caused_by": {
            "type": "mustache_exception",
            "reason": "Failed to get value for ctx.payload.aggregations.bucket_results.top_bucket_hits.hits.hits.0._source.job_id @[query-template:1]",
            "caused_by": {
              "type": "mustache_exception",
              "reason": "0 @[query-template:1]",
              "caused_by": {
                "type": "index_out_of_bounds_exception",
                "reason": "0"
              }
            }
          }
        }
      },
      {
        "id": "slack_2",
        "type": "slack",
        "status": "failure",
        "error": {
          "root_cause": [
            {
              "type": "general_script_exception",
              "reason": "Error running com.github.mustachejava.codes.DefaultMustache@53983561"
            }
          ],
          "type": "general_script_exception",
          "reason": "Error running com.github.mustachejava.codes.DefaultMustache@53983561",
          "caused_by": {
            "type": "mustache_exception",
            "reason": "Failed to get value for ctx.payload.aggregations.bucket_results.top_bucket_hits.hits.hits.0._source.job_id @[query-template:11]",
            "caused_by": {
              "type": "mustache_exception",
              "reason": "0 @[query-template:11]",
              "caused_by": {
                "type": "index_out_of_bounds_exception",
                "reason": "0"
              }
            }
          }
        }
      }
    ]
  },
  "messages": []
}
2

If info is needed about the watcher let me know what you wanna see because I can not post the whole watcher (it has more than 7000 characters)

3

Hi @sonnydoza - just use an online pastebin or GitHub gist or similar to post the full syntax of your watcher. Redact private info as necessary.

4

Thank you for the response Rich.

There it goes https://gist.github.com/sonnydoza/1ee02261e051eeeee40a29e5a2f074ed

I created the watcher through the UI the only changes that I made are the actions to make it work with Slack and also the "size" in the body input.

5

I just copied and ran your exact watch (except I took out the slack piece) and it ran just fine.

Copy the entire JSON from your watch, then go over to the DevTools console. In there, put the following:

POST _xpack/watcher/watch/_execute
{
  "watch" :  

<paste your entire watch JSON here>
}

and execute it (this is pretty much what the simulate does, but you don't need to formally define the watch first).

Do you still get the error?

1 Like
6

Good morning Rich.

I did run the watcher in the dev tools as you told me and it triggers the actions successfully and it sends the message to the slack channel.

Any idea why this happens?

7

So, given the information reported, it seems like you're all set - might have just been a fluke problem when you were trying to simulate it in the Watcher UI.

Let the watch run as scheduled and see if you occasionally get an alert notification if/when an anomaly occurs.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.