Watcher Basic Alert

Elastic Stack Elasticsearch
1

I'm trying to schedule a cluster health alert

curl -k -u usr:pwd-XPUT 'https://localhost:9200/_watcher/watch/cluster_red_alert' -d ' {
"trigger" : {
"schedule" : { "interval" : "10s" }
},
"input" : {
"http" : {
"request" : {
"host" : "localhost",
"port" : 9200,
"path" : "/_cluster/health"
}
}
}
}'

But this is giving me the error :

"messages" : [ "failed to execute watch input" ],
"result" : {
"execution_time" : "2016-06-09T22:41:03.551Z",
"execution_duration" : 3,
"input" : {
"type" : "http",
"status" : "failure",
"reason" : "SocketException[Unexpected end of file from server]",
"http" : {
"request" : {
"host" : "localhost",
"port" : 9200,
"scheme" : "http",
"method" : "get",
"path" : "/_cluster/health"
}
}
},

ES 2.3.1
I have shield installed.

2

Hey,

you are trying to execute a HTTP request against a TLS/SSL secured resource, and therefore elasticsearch cuts the connection, as it does not receive the expected TLS data.

  • Use HTTPS, by specifying "scheme" : "https" in the request. The http input does not know if you are using shield or not, it is basically just a HTTP client and thus not very smart
  • If you are using shield it is likely that you are using auth, which requires you to add auth information as well
"request" : {
      ...
      "auth" : {
        "basic" : {
          "username" : "user",
          "password" : "pass"
        }
      }

Hope this helps. Also, please use code blocks to show your watch, it makes it incredibly easier for others to read.

--Alex

3

Thanks Alex,

Can't believe I missed that :grin:

do you a have sample watch that works with shield? or do you know where I can find one?

I got this error when I'm trying, if there was an example I can leverage.

{"error":{"root_cause":[{"type":"parse_exception","reason":"could not read search request. unexpected string field [scheme]"}],"type":"parse_exception","reason":"could not parse [search] input for watch [cluster_red_alert]. failed to parse [request]","caused_by":{"type":"parse_exception","reason":"could not read search request. unexpected string field [scheme]"}},"status":400}

4

If I'm running watcher on the same cluster which I'm watching, would I still need creds?

5

Hey,

see the almighty doucmentation with an auth example - if you are using the HTTP input, you will have to provide your credentials.

You might also be interested in the attributes for the request in the search input

Hope this helps.

--Alex

6

Thanks Alex,

It did help.

Now I have the alert created. I see this error in the result.condition.reason column in kibana. When I'm monitoring it in Kibana.

ScriptException[failed to run inline script [if (ctx.payload.hits.total < 1) return false; def rows = ctx.payload.hits.hits; if (rows[0].fields[cluster_state.status][0] != red) return false; if (ctx.payload.aggregations.minutes.buckets.size() < 12) return false; def last60Seconds = ctx.payload.aggregations.minutes.buckets[-12..-1]; return last60Seconds.every { it.status.buckets.every { s -> s.key == red}}] using lang [groovy]]; nested: MissingPropertyException[No such property: cluster_state for class: a24fecec8b3ef0eeb9944ffefb62dce99822bbab];

7

Hey,

apparently you are trying to access a property that does not exist, which is named cluster_state. Why are you using fields and not the _source field? Also cluster_state.status is kind of a shortcut for two elements, and it is not put in quotes, thus interpreted as a variable.

--Alex

8

I was using the monitor the marvel data watch from the elasticsearch website.

https://www.elastic.co/guide/en/watcher/current/watching-marvel-data.html#watching-cluster-health

Only change made was converting into curl command

9

hey,

I just tested the example under https://www.elastic.co/guide/en/watcher/current/watching-marvel-data.html#watching-cluster-health and replaced red with yellow for testing purposes and it triggered to send an email.

Can you paste the output of

PUT _watcher/watch/cluster_red_alert/_execute

Which Elasticsearch version are you using?

--Alex

10

thanks Alex. they updated the documentation yesterday...support confirmed
that. also curl was stripping off the quotes from my code. it is working
when I do a json import