# Watcher Alert mail Issue **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-alerting\ **Created:** [November 4, 2015, 6:59am UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706 "2015-11-04T06:59:42Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [November 4, 2015, 6:59am UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/1 "2015-11-04T06:59:42Z")
Hi Team, I have created new watcher PUT /\_watcher/watch/cluster\_health\_watch { "trigger" : { "schedule" : { "interval" : "30s" } }, "input" : { "http" : { "request" : { "host" : "localhost", "port" : port, "path" : "/\_cluster/health" } } }, "condition" : { "compare" : { "ctx.payload.status" : { "eq" : "green" } } }, "actions" : { "send\_email" : { "email" : { "to" : "ganeshbabu.ramamoorthy@company.com", "subject" : "Cluster Status Health", "body" : "Cluster status is Green" } } } } When I checked watcher history i am getting the following results, { "took": 61, "timed\_out": false, "\_shards": { "total": 1, "successful": 1, "failed": 0 }, "hits": { "total": 27, "max\_score": 1, "hits": [ { "\_index": ".watch\_history-2015.11.03", "\_type": "watch\_record", "\_id": "cluster\_health\_watch\_0-2015-11-03T07:18:09.701Z", "\_score": 1, "\_source": { "watch\_id": "cluster\_health\_watch", "state": "execution\_not\_needed", "trigger\_event": { "type": "schedule", "triggered\_time": "2015-11-03T07:18:09.701Z", "schedule": { "scheduled\_time": "2015-11-03T07:18:09.395Z" } }, "input": { "http": { "request": { "scheme": "http", "host": "localhost", "port": port, "method": "get", "path": "/\_cluster/health", "params": {}, "headers": {} } } }, "condition": { "compare": { "ctx.payload.status": { "eq": "green" } } }, "messages": [], "result": { "execution\_time": "2015-11-03T07:18:09.701Z", "execution\_duration": 105, "input": { "type": "http", "status": "success", "payload": {}, "http": { "request": { "host": "localhost", "port": port, "scheme": "http", "method": "get", "path": "/\_cluster/health" }, "status\_code": 401 } }, "condition": { "type": "compare", "status": "success", "met": false, "compare": { "resolved\_values": { "ctx.payload.status": null } } }, "actions": [] } } }, Why **ctx.payload.status become "null"** and action block shown as empty. But I can see the count is increasing based on the interval(30s). Why I didn't any alert mail from the server. Is watcher will only work for negative scenario but I created watcher with positive scenario will it work? Please help me to resolve this issue. I took the below link as reference to create watcher [https://www.elastic.co/guide/en/watcher/current/watch-cluster-status.html](https://www.elastic.co/guide/en/watcher/current/watch-cluster-status.html) Regards, Ganeshbabu R ---
**Author:** ![SuperPringles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/superpringles/32/5303_2.png) [@SuperPringles](https://discuss.elastic.co/u/SuperPringles)\ **Post date:** [November 4, 2015, 7:49am UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/2 "2015-11-04T07:49:05Z")
I'm by no way an expert in this but you gave the port the value port. Wouldn't it now look for localhost:port ? I once tried making it localhost2:9200 and it still showed up in Kibana but it never did anything of use. So maybe you should give it a proper port. ---
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [November 4, 2015, 10:40am UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/3 "2015-11-04T10:40:09Z")
Thanks for your reponse @SuperPringles I tried by setting up the host as " **localhost**" but still " **Payload**" block returns **no response** and the **status becomes "null"** **Sample Watcher"** PUT /\_watcher/watch/cluster\_health\_watch1 { "trigger" : { "schedule" : { "interval" : "30s" } }, "input" : { "http" : { "request" : { "host" : "localhost", "port" : 9200, "path" : "/\_cluster/health" } } }, "condition" : { "compare" : { "ctx.payload.status" : { "eq" : "green" } } }, "actions" : { "send\_email" : { "email" : { "to" : "ganeshbabu.ramamoorthy@company.com", "subject" : "Cluster Status Health", "body" : "Cluster status is Green" } } } } **Sample output:-** { "\_index": ".watch\_history-2015.11.04", "\_type": "watch\_record", "\_id": "cluster\_health\_watch1\_0-2015-11-04T09:49:45.617Z", "\_score": 2.6563215, "\_source": { "watch\_id": "cluster\_health\_watch1", "state": "execution\_not\_needed", "trigger\_event": { "type": "schedule", "triggered\_time": "2015-11-04T09:49:45.617Z", "schedule": { "scheduled\_time": "2015-11-04T09:49:45.137Z" } }, "input": { "http": { "request": { "scheme": "http", "host": "localhost", "port": 9200, "method": "get", "path": "/\_cluster/health", "params": {}, "headers": {} } } }, "condition": { "compare": { "ctx.payload.status": { "eq": "green" } } }, "messages": [], "result": { "execution\_time": "2015-11-04T09:49:45.617Z", "execution\_duration": 54, "input": { "type": "http", "status": "success", **"payload": {},** "http": { "request": { "host": "localhost", "port": 9200, "scheme": "http", "method": "get", "path": "/\_cluster/health" }, "status\_code": 401 } }, "condition": { "type": "compare", "status": "success", "met": false, "compare": { "resolved\_values": { "ctx.payload.status": null } } }, "actions": [] } } }, Why status\_code:- 401? Is it because of not given authentication details. Can any one help me to resolve this/ Thanks, Ganeshbabu R ---
**Author:** ![SuperPringles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/superpringles/32/5303_2.png) [@SuperPringles](https://discuss.elastic.co/u/SuperPringles)\ **Post date:** [November 4, 2015, 11:07am UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/4 "2015-11-04T11:07:19Z")
This would be an example of how I make my watcher. As you can see I don't add authentication details either. Is it possible that your setup doesn't have enough permissions? ``` curl -XPUT 'http://localhost:9200/_watcher/watch/cluster_health_watch3' -d '{ "trigger" : { "schedule" : { "interval" : "10s" } }, "input" : { "http" : { "request" : { "host" : "localhost", "port" : 9200, "path" : "/_cluster/health" } } }, "condition" : { "compare" : { "ctx.payload.status" : { "eq" : "green" } } }, "actions" : { "send_email" : { "email" : { "to" : "myemail@gmail.com", "subject" : "Cluster Status Warning", "body" : "Cluster status is YELLOW" } } } }' ``` I also notice how your "state" returns "execution\_not\_needed" This means it did not deem it necessary to execute it. This is shown when it compared but it wasn't the same value. I get this when I look for a "red" status when my target is in the "yellow". I do get a payload however ``` "payload": { "cluster_name": "elasticsearch", "status": "yellow", "timed_out": false, "number_of_nodes": 1, "number_of_data_nodes": 1, "active_primary_shards": 17, "active_shards": 17, "relocating_shards": 0, "initializing_shards": 0, "unassigned_shards": 17, "delayed_unassigned_shards": 0, "number_of_pending_tasks": 0, "number_of_in_flight_fetch": 0 }, ``` I suspect it can't show you this information because it doesn't have access to them which is probably why you're getting a 401. Like I said, maybe there is something wrong with the permissions? I hope this helps you? I'm pretty much new to ubuntu and elastic all together but I want to help you either way 😄 ---
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [November 4, 2015, 1:27pm UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/5 "2015-11-04T13:27:31Z")
Thanks for your response @SuperPringles Without authentication I cannot create watcher using curl command. It will show authentication exception **"error":"AuthenticationException[missing authentication token for REST request [/\_watcher/watch/cluster\_health\_watch1** because I installed **shield plugin** in elasticsearch, So Its expecting authentication details to do some operations. But still i get the same above error **"Payload"** block returns no response and the **status becomes "null"** Similarly I created the same type of **watcher** with **search type request.** PUT /\_watcher/watch/item\_watch { "trigger": { "schedule": { "interval": "1m" } }, "input": { "search": { "request": { "indices": [ "item" ], "body": { "query": { "match": { "ITEM\_ID": 2746068 } } } } } }, "condition": { "compare": { "ctx.payload.hits.total": { "gt": 0 } } }, "actions": { "email\_admin": { "email": { "to": "ganeshbabu.ramamoorthy@company.com", "subject": "{{ctx.watch\_id}} executed", "body": "{{ctx.watch\_id}} executed with {{ctx.payload.hits.total}} hits" } } } } But for the search type request watcher I am getting proper response like getting alert mail. I don't know why It's not working for **http** request. I am just following the documentation given in the elasticsearch watcher. I need to find it out help from others. Thanks, Ganeshbabu R ---
**Author:** ![SuperPringles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/superpringles/32/5303_2.png) [@SuperPringles](https://discuss.elastic.co/u/SuperPringles)\ **Post date:** [November 4, 2015, 2:07pm UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/6 "2015-11-04T14:07:08Z")
I haven't used Shield so I can only imagine but is it possible that Shield is blocking HTTP requests as part of its protection? ---
**Author:** ![skearns](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/skearns/32/125945_2.png) [@skearns](https://discuss.elastic.co/u/skearns)\ **Post date:** [November 4, 2015, 3:04pm UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/7 "2015-11-04T15:04:59Z")
Hi Ganeshbabu, If you are querying against the same cluster that Watcher is installed on, you should be using the 'search' input instead of the HTTP input. The search input will execute the query locally, and won't require username/password config. If you are querying a remote cluster over HTTP(s), and that remote cluster is protected by Shield, you will have to include authentication information as part of the Watcher HTTP input. [https://www.elastic.co/guide/en/watcher/current/input.html#input-search](https://www.elastic.co/guide/en/watcher/current/input.html#input-search) Thanks, Steve ---
**Author:** ![SuperPringles](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/superpringles/32/5303_2.png) [@SuperPringles](https://discuss.elastic.co/u/SuperPringles)\ **Post date:** [November 4, 2015, 3:06pm UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/8 "2015-11-04T15:06:40Z")
Not entirely as I had suspected but I was pretty close 😄 ---
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [November 4, 2015, 5:26pm UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/9 "2015-11-04T17:26:16Z")
Thanks for your response @skearns Suppose If I am querying against the same cluster over HTTP and the watcher is installed on. My understanding is 1. the watcher will not finds the cluster health. 2. Only external elasticsearch clusters will help the HTTP to find the cluster health Is it right? Regards, Ganeshbabu R ---
**Author:** ![r.ganeshbabu](https://avatars.discourse-cdn.com/v4/letter/r/4da419/32.png) [@r.ganeshbabu](https://discuss.elastic.co/u/r.ganeshbabu)\ **Post date:** [December 15, 2015, 6:34am UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/10 "2015-12-15T06:34:05Z")
Hi all, Below is the Watcher Email configuration from elasticsearch.yml, ################################## Watcher ################################# **watcher.actions.email.service.account:** es\_account: profile: standard smtp: auth: false starttls.enable: true host: [smarthost.enterprise.com](http://smarthost.enterprise.com) port: 25 user: password: I have created watcher to find the cluster\_health, PUT /\_watcher/watch/cluster\_health\_watch { "trigger" : { "schedule" : { "interval" : "60s" } }, "input" : { "http" : { "request" : { "host" : "10.7.146.26", "port" : 9200, "path" : "/\_cluster/health", "auth": { "basic": { "username": "esadmin", "password": "password" } } } } }, "condition" : { "compare" : { "ctx.payload.status" : { "eq" : "green" } } }, "actions" : { "email\_administrator" : { "email" : { "to" : "ganeshbabu.ramamoorthy.ap@company.com", "subject" : "Cluster Status Health", "body" : "Cluster status is Green", "attach\_data" : true, "priority" : "high" } } } } I configured watcher email setup in master node only and below is the error response from master log during watcher execution. **Note:- Still I can get watcher alert mail from the server even though the error showing in master log.** [2015-12-15 01:21:06,085][ERROR][watcher.actions.email] [dayrhed001\_DEV\_MASTER] failed to execute action [cluster\_health\_watch/email\_administrator] org.elasticsearch.watcher.actions.email.service.EmailException: failed to send email with subject [Cluster Status Health] via account [es\_account] at org.elasticsearch.watcher.actions.email.service.InternalEmailService.send(InternalEmailService.java:86) at org.elasticsearch.watcher.actions.email.service.InternalEmailService.send(InternalEmailService.java:78) at org.elasticsearch.watcher.actions.email.ExecutableEmailAction.execute(ExecutableEmailAction.java:67) at org.elasticsearch.watcher.actions.ActionWrapper.execute(ActionWrapper.java:104) at org.elasticsearch.watcher.execution.ExecutionService.executeInner(ExecutionService.java:379) at org.elasticsearch.watcher.execution.ExecutionService.execute(ExecutionService.java:271) at org.elasticsearch.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:417) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) **Caused by: javax.mail.AuthenticationFailedException: No authentication mechanisms supported by both server and client** at com.sun.mail.smtp.SMTPTransport.authenticate(SMTPTransport.java:818) at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:728) at javax.mail.Service.connect(Service.java:364) at org.elasticsearch.watcher.actions.email.service.Account.send(Account.java:99) at org.elasticsearch.watcher.actions.email.service.InternalEmailService.send(InternalEmailService.java:84) Please guide me to resolve this error. Thanks Ganeshbabu R ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:47pm UTC](https://discuss.elastic.co/t/watcher-alert-mail-issue/33706/11 "2017-07-06T13:47:48Z")