# Failed to start watcher. please wait for the cluster to become ready or try to start Watcher manully
**URL:** https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669
**Category:** Elasticsearch
**Tags:** elastic-stack-alerting
**Created:** [September 28, 2016, 7:38am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669 "2016-09-28T07:38:34Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@sahilarora0928](https://discuss.elastic.co/u/sahilarora0928)
#### Post date: [September 28, 2016, 7:38am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/1 "2016-09-28T07:38:34Z")
Hi,
I have installed watcher plugin, trying to run elastic search but getting exception.
Please help me to resolve this. Its urgent.
Thanks,
Sahil
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [September 28, 2016, 7:46am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/2 "2016-09-28T07:46:31Z")
Hey,
please paste logs by text and not images and also check if there are other exceptions on startup. What happens when you actually start watcher manually as mentioned in the error message?
--Alex
---
### Author:  [@sahilarora0928](https://discuss.elastic.co/u/sahilarora0928)
#### Post date: [September 28, 2016, 8:28am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/3 "2016-09-28T08:28:55Z")
D:\elasticsearch\elasticsearch-2.4.0\elasticsearch-2.4.0\bin\>elasticsearch
[2016-09-28 13:45:21,569][INFO][node] [Hideko Takata] version[2.4.0], pid[3532], build[ce9f0c7/2016-08-29T09:14:17Z]
[2016-09-28 13:45:21,570][INFO][node] [Hideko Takata] initializing ...
[2016-09-28 13:45:22,051][INFO][plugins] [Hideko Takata] modules [lang-groovy, reindex, lang-expression], plugins [watcher, license], sites []
[2016-09-28 13:45:22,085][INFO][env] [Hideko Takata] using [1] data paths, mounts [[(D:)]], net usable\_space [290.2gb], net total\_space [319.2gb], spins
? [unknown], types [NTFS]
[2016-09-28 13:45:22,087][INFO][env] [Hideko Takata] heap size [910.5mb], compressed ordinary object pointers [true]
[2016-09-28 13:45:22,115][INFO][watcher.trigger.schedule] [Hideko Takata] using [ticker] schedule trigger engine
[2016-09-28 13:45:25,128][INFO][node] [Hideko Takata] initialized
[2016-09-28 13:45:25,128][INFO][node] [Hideko Takata] starting ...
[2016-09-28 13:45:25,481][INFO][transport] [Hideko Takata] publish\_address {127.0.0.1:9300}, bound\_addresses {[::1]:9300}, {127.0.0.1:9300}
[2016-09-28 13:45:25,484][INFO][discovery] [Hideko Takata] elasticsearch/wSWMrJhFR7qVKF4UzrtjaQ
[2016-09-28 13:45:29,520][INFO][cluster.service] [Hideko Takata] new\_master {Hideko Takata}{wSWMrJhFR7qVKF4UzrtjaQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-jo
in(elected\_as\_master, [0] joins received)
[2016-09-28 13:45:29,878][INFO][http] [Hideko Takata] publish\_address {127.0.0.1:9200}, bound\_addresses {[::1]:9200}, {127.0.0.1:9200}
[2016-09-28 13:45:30,528][INFO][license.plugin.core] [Hideko Takata] license [536f535b-d32d-4d2c-aa05-e66e4c8f8adc] - valid
[2016-09-28 13:45:33,059][INFO][node] [Hideko Takata] started
[2016-09-28 13:45:33,061][ERROR][license.plugin.core] [Hideko Takata]
#
# License will expire on [Wednesday, October 26, 2016]. If you have a new license, please update it.
# Otherwise, please reach out to your support contact.
#
# Commercial plugins operate with reduced functionality on license expiration:
# - watcher
# - PUT / GET watch APIs are disabled, DELETE watch API continues to work
# - Watches execute and write to the history
# - The actions of the watches don't execute
[2016-09-28 13:45:40,909][INFO][gateway] [Hideko Takata] recovered [174] indices into cluster\_state
[2016-09-28 13:45:44,601][INFO][watcher] [Hideko Takata] starting watch service...
[2016-09-28 13:46:44,652][WARN][watcher] [Hideko Takata] failed to start watcher. please wait for the cluster to become ready or try to start Watcher manual
ly
ElasticsearchTimeoutException[Timeout waiting for task.]
at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:70)
at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:62)
at org.elasticsearch.watcher.support.init.proxy.ClientProxy.index(ClientProxy.java:86)
at org.elasticsearch.watcher.history.HistoryStore.forcePut(HistoryStore.java:115)
at org.elasticsearch.watcher.execution.ExecutionService.executeTriggeredWatches(ExecutionService.java:404)
at org.elasticsearch.watcher.execution.ExecutionService.start(ExecutionService.java:99)
at org.elasticsearch.watcher.WatcherService.start(WatcherService.java:82)
at org.elasticsearch.watcher.WatcherLifeCycleService.start(WatcherLifeCycleService.java:100)
at org.elasticsearch.watcher.WatcherLifeCycleService.access$100(WatcherLifeCycleService.java:36)
at org.elasticsearch.watcher.WatcherLifeCycleService$3.run(WatcherLifeCycleService.java:151)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2016-09-28 13:46:44,705][INFO][watcher] [Hideko Takata] starting watch service...
[2016-09-28 13:47:44,730][WARN][watcher] [Hideko Takata] failed to start watcher. please wait for the cluster to become ready or try to start Watcher manual
ly
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [September 28, 2016, 8:50am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/4 "2016-09-28T08:50:40Z")
Hey,
my current assumption is, that the watch history index has not recovered. You can use the [recovery API](https://www.elastic.co/guide/en/elasticsearch/reference/2.4/cat-recovery.html) to find out and use [watcher stats](https://www.elastic.co/guide/en/watcher/2.4/api-rest.html#api-rest-stats) to find out if watcher is started?
Also, did you start watcher manually as suggested in the logs and in my last post?
--Alex
---
### Author:  [@sahilarora0928](https://discuss.elastic.co/u/sahilarora0928)
#### Post date: [September 28, 2016, 9:28am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/5 "2016-09-28T09:28:51Z")
Hi Alex,
I deleted the watch indexes and restart elastic search again and it worked.
Here I am trying to send an alert using gmail but i am getting exception for that as well (failed to update the watch record and connection refused). Check below
```
... 13 more
```
**[2016-09-28 14:52:02,819][ERROR][watcher.execution] [Vavavoom] failed to update watch record [event\_critical\_watch\_25-2016-09-28T09:20:32.141Z]**
ElasticsearchTimeoutException[Timeout waiting for task.]
at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:70)
at org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:62)
at org.elasticsearch.watcher.support.init.proxy.ClientProxy.index(ClientProxy.java:86)
at org.elasticsearch.watcher.history.HistoryStore.put(HistoryStore.java:93)
at org.elasticsearch.watcher.execution.ExecutionService.execute(ExecutionService.java:302)
at org.elasticsearch.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:438)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2016-09-28 14:52:03,919][ERROR][watcher.actions.email] [Vavavoom] failed to execute action [event\_critical\_watch/email\_admin]
javax.mail.MessagingException: failed to send email with subject [event\_critical\_watch executed] via account [gmail];
nested exception is:
**com.sun.mail.util.MailConnectException: Couldn't connect to host, port: [smtp.gmail.com](http://smtp.gmail.com), 587; timeout 120000;**
\*\* nested exception is:\*\*
\*\* java.net.ConnectException: Connection refused: connect\*\*
at org.elasticsearch.watcher.actions.email.service.InternalEmailService.send(InternalEmailService.java:85)
at org.elasticsearch.watcher.actions.email.service.InternalEmailService.send(InternalEmailService.java:77)
at org.elasticsearch.watcher.actions.email.ExecutableEmailAction.execute(ExecutableEmailAction.java:84)
at org.elasticsearch.watcher.actions.ActionWrapper.execute(ActionWrapper.java:106)
at org.elasticsearch.watcher.execution.ExecutionService.executeInner(ExecutionService.java:388)
at org.elasticsearch.watcher.execution.ExecutionService.execute(ExecutionService.java:273)
at org.elasticsearch.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:438)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.sun.mail.util.MailConnectException: Couldn't connect to host, port: [smtp.gmail.com](http://smtp.gmail.com), 587; timeout 120000;
nested exception is:
java.net.ConnectException: Connection refused: connect
at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2054)
at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:697)
at javax.mail.Service.connect(Service.java:364)
at org.elasticsearch.watcher.actions.email.service.Account.send(Account.java:124)
at org.elasticsearch.watcher.actions.email.service.InternalEmailService.send(InternalEmailService.java:83)
... 9 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
**This is my SMTP configuration in elasticsearch.yml**
watcher.actions.email.service.account:
gmail:
profile: gmail
smtp:
auth: true
starttls.enable: true
host: [smtp.gmail.com](http://smtp.gmail.com)
port: 587
user: \*\*\*\*\*\*\*\*\*@gmail.com
password: \*\*\*\*\*\*\*\*\*\*\*\*
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [September 28, 2016, 9:46am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/6 "2016-09-28T09:46:07Z")
Hey,
it does not look to me as if your cluster recovered fully and back to a good state, as it cannot write into the watch history index. Did you run the cat API I mentioned in a previous blog post?
On top of that, it seems you cannot reach gmail from your network.
--Alex
---
### Author:  [@sahilarora0928](https://discuss.elastic.co/u/sahilarora0928)
#### Post date: [September 28, 2016, 11:36am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/7 "2016-09-28T11:36:25Z")
Running cat API means this ([http://localhost:9200/\_cat/indices?v](http://localhost:9200/_cat/indices?v) ) right ? After running this I got the the health status of every index as yellow .
A small snapshot of that is here.
Is this correct or shall i delete all the indices and start fresh ..
Also please share the link to previous blog post where you have mentioned about this all. I couldnt find it.
Please let me know .
Thanks.
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [September 29, 2016, 7:24am UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/8 "2016-09-29T07:24:35Z")
I was referring to this very thread, where I mentioned which cat API output would help here - so please see my above replies.
Also, please refrain from using screenshots, but paste the text output and paste the full output and not just parts. Otherwise it is so much harder to help.
--Alex
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:42pm UTC](https://discuss.elastic.co/t/failed-to-start-watcher-please-wait-for-the-cluster-to-become-ready-or-try-to-start-watcher-manully/61669/9 "2017-07-06T13:42:44Z")