# Elasticsearch errors
**URL:** https://discuss.elastic.co/t/elasticsearch-errors/89720
**Category:** Elasticsearch
**Created:** [June 16, 2017, 1:08pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720 "2017-06-16T13:08:21Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@raghvendra](https://discuss.elastic.co/u/raghvendra)
#### Post date: [June 16, 2017, 1:08pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/1 "2017-06-16T13:08:21Z")
I ..I have 3 node of cluster . I have installed x pack . Everything was working fine but by mistake , one node of my cluster got disconnected . So I restarted cluster .. now it is giving these errors .
> [2017-06-16T18:13:35,340][ERROR][o.e.x.m.c.i.IndicesStatsCollector] [slave-1] collector [indices-stats] failed to collect data
> org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE\_UNAVAILABLE/1/state not recovered / initialized];
After few exceptions -
>
[2017-06-16T18:13:35,344][ERROR][o.e.x.m.c.i.IndexRecoveryCollector] [slave-1] collector [index-recovery] failed to collect data
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE\_UNAVAILABLE/1/state not recovered / initialized];
then
> [2017-06-16T18:13:35,350][ERROR][o.e.x.m.c.i.IndexStatsCollector] [slave-1] collector [index-stats] failed to collect data
> org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE\_UNAVAILABLE/1/state not recovered / initialized];
and
> [2017-06-16T18:14:03,422][ERROR][o.e.x.m.e.l.LocalExporter] failed to get monitoring watch [8ix4VdBmSJOml0PwEdjt8w\_elasticsearch\_cluster\_status]
> java.lang.IllegalStateException: watch store not started
then
> [2017-06-16T18:14:03,426][ERROR][o.e.x.w.t.a.g.TransportGetWatchAction] [slave-1] failed to get watch [8ix4VdBmSJOml0PwEdjt8w\_elasticsearch\_version\_mismatch]
> java.lang.IllegalStateException: watch store not started
then
> [2017-06-16T18:14:03,429][ERROR][o.e.x.m.e.l.LocalExporter] failed to get monitoring watch [8ix4VdBmSJOml0PwEdjt8w\_elasticsearch\_version\_mismatch]
> java.lang.IllegalStateException: watch store not started
and so on. I had deleted the previous index before and then restarted it .. Since then , it is giving this .. even though cluster is working .. only monitoring section is not accessible .
Can anyone help me with this ?
---
### Author:  [@tsullivan](https://discuss.elastic.co/u/tsullivan)
#### Post date: [June 19, 2017, 8:50pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/2 "2017-06-19T20:50:28Z")
When you say "Monitoring section is not accessible" -- what do you get when you try to access Monitoring?
Can you share the output of a search in some Monitoring data?
```auto
GET /.monitoring-data-2/cluster_info/_search?filter_path=hits.hits._source.cluster_uuid,hits.hits._source.license.status
```
---
### Author:  [@raghvendra](https://discuss.elastic.co/u/raghvendra)
#### Post date: [June 20, 2017, 5:38am UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/3 "2017-06-20T05:38:25Z")
yeah .. sometimes it works fine , sometimes it shows this -\>
---
### Author:  [@tsullivan](https://discuss.elastic.co/u/tsullivan)
#### Post date: [June 22, 2017, 9:07pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/4 "2017-06-22T21:07:11Z")
Can you check that your clusters have valid licenses?
I edited the query that I pasted above so that I'm not asking you to share private info - the query just gets cluster UUID and license status from the monitoring data. Can you run that query so we can check if this is a license issue?
---
### Author:  [@raghvendra](https://discuss.elastic.co/u/raghvendra)
#### Post date: [June 23, 2017, 5:38am UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/5 "2017-06-23T05:38:40Z")
Sorry for the late reply .
Btw It is showing license is active .
---
### Author:  [@tsullivan](https://discuss.elastic.co/u/tsullivan)
#### Post date: [June 26, 2017, 10:00pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/6 "2017-06-26T22:00:29Z")
Do you have any templates that might have patterns which clobber the monitoring data?
Try the `http://localhost:9200/_template/` API and look for templates with index patterns that possibly match match monitoring indices.
---
### Author:  [@raghvendra](https://discuss.elastic.co/u/raghvendra)
#### Post date: [June 28, 2017, 9:43am UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/7 "2017-06-28T09:43:46Z")
Hi..
I tried [http://localhost:9200/\_template/](http://localhost:9200/_template/)
Instead of localhost , I had to use IP of the node and I got this;
---
### Author:  [@tsullivan](https://discuss.elastic.co/u/tsullivan)
#### Post date: [June 28, 2017, 4:45pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/8 "2017-06-28T16:45:48Z")
Have you added any of your own templates to Elasticsearch?
If so, should check that you haven't created a template with an index pattern that might match the `.monitoring*` indices.
You should paste your template output into a JSON formatter and look at each template's index pattern to make sure it wouldn't mistakenly match the `.monitoring*` indices.
---
### Author:  [@raghvendra](https://discuss.elastic.co/u/raghvendra)
#### Post date: [June 29, 2017, 12:48pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/9 "2017-06-29T12:48:52Z")
Hey .. sorry for the late response .
I haven't created my own template , I am going on with the default configuration till now .
I do want to mention that the kibana monitoring section some times works , sometimes doesn't . Right now , it's working .
In fact, I haven't faced this error for 3-4 days . I have changed nothing , I don't know why there's no error.
As far as , I remember , when I was having this error , perhaps , there was some logs in elasticsearch console , indicating , failed to start monitoring shards or something like this.
Unfortunately , I couldn't take screenshot , and I have also deleted the old logs in elasticsaerch log folder
Next time , if it occurs , I'll share the screenshot with you in PM.
Thanks
---
### Author:  [@tsullivan](https://discuss.elastic.co/u/tsullivan)
#### Post date: [June 30, 2017, 8:58pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/10 "2017-06-30T20:58:41Z")
Can I make a hypothesis and guess that when the error happens, the time picker date range was expanded to include a time that the cluster had a different license type?
I'll continue to investigate. Stay tuned.
---
### Author:  [@raghvendra](https://discuss.elastic.co/u/raghvendra)
#### Post date: [July 1, 2017, 5:46am UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/11 "2017-07-01T05:46:45Z")
No, I don't think so , I always use last 2 years in time picker . If it is a license issue then, it should be behaving monotonically either it would work or it won't . No combination of both cases can happen .
---
### Author:  [@pmisra19](https://discuss.elastic.co/u/pmisra19)
#### Post date: [July 20, 2017, 3:09pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/12 "2017-07-20T15:09:00Z")
I was just doing a POC and noticed on my elastic search today that i was getting a similar error. I think mine has to do with license issue. i ran the below with the results posted below:
GET /.monitoring-data-2/cluster\_info/\_search?filter\_path=hits.hits.\_source.cluster\_uuid,hits.hits.\_source.license.status
result:
{
"hits": {
"hits": [
{
"\_source": {
"cluster\_uuid": "ZaAaQ17FQbuQtfzIxFpO6g",
"license": {
"status": "active"
}
}
}
]
}
}
Log stack :
[ERROR][o.e.x.w.t.a.g.TransportGetWatchAction] [8NX03Oq] failed to get watch [ZaAaQ17FQbuQtfzIxFpO6g\_elasticsearch\_cluster\_status]
java.lang.IllegalStateException: watch store not started
at org.elasticsearch.xpack.watcher.watch.WatchStore.ensureStarted(WatchStore.java:367) ~[?:?]
at org.elasticsearch.xpack.watcher.watch.WatchStore.get(WatchStore.java:153) ~[?:?]
at org.elasticsearch.xpack.watcher.WatcherService.getWatch(WatcherService.java:158) ~[?:?]
at org.elasticsearch.xpack.watcher.transport.actions.get.TransportGetWatchAction.masterOperation(TransportGetWatchAction.java:86) ~[?:?]
at org.elasticsearch.xpack.watcher.transport.actions.get.TransportGetWatchAction.masterOperation(TransportGetWatchAction.java:53) ~[?:?]
at org.elasticsearch.action.support.master.TransportMasterNodeAction.masterOperation(TransportMasterNodeAction.java:87) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$2.doRun(TransportMasterNodeAction.java:166) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:110) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.doStart(TransportMasterNodeAction.java:163) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction.start(TransportMasterNodeAction.java:126) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.action.support.master.TransportMasterNodeAction.doExecute(TransportMasterNodeAction.java:104) ~[elasticsearch-5.4.2.jar:5.4.2]
at org.elasticsearch.xpack.watcher.transport.actions.WatcherTransportAction.doExecute(WatcherTransportAction.java:56) ~[?:?]
at org.elasticsearch.xpack.watcher.transport.actions.WatcherTransportAction.doExecute(WatcherTransportAction.java:40) ~[?:?]
---
### Author:  [@dvmrp](https://discuss.elastic.co/u/dvmrp)
#### Post date: [July 27, 2017, 9:47am UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/13 "2017-07-27T09:47:17Z")
I am running into the same situation.
---
### Author:  [@anitakv](https://discuss.elastic.co/u/anitakv)
#### Post date: [August 3, 2017, 9:51pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/14 "2017-08-03T21:51:59Z")
DId you get to resolve this ? I get the same problem. I am actually trying to install both logstash and elasticsearch. Elastic search logs fail with
[2017-08-03T21:42:02,543][ERROR][o.e.x.m.e.l.LocalExporter] failed to get monitoring watch [aW-uiux4TDm7PODVVN6ZIg\_logstash\_version\_mismatch]
java.lang.IllegalStateException: watch store not started
at org.elasticsearch.xpack.watcher.watch.WatchStore.ensureStarted(WatchStore.java:367) ~[?:?]
---
### Author:  [@tsullivan](https://discuss.elastic.co/u/tsullivan)
#### Post date: [August 4, 2017, 12:05am UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/15 "2017-08-04T00:05:31Z")
It seems that the `failed to get watch` errors and `watch store not started` can be safely ignored. What that means is that the Monitoring plugin for Elasticsearch is trying to add watches, but Elasticsearch isn't initialized far enough for that to work. The Monitoring plugin will keep trying and adding the watch should work the next time.
If you are concerned that Watcher is not working in the cluster or that the Monitoring plugin is getting stuck trying to send the watches, you could try manually executing the watches by ID. The ID of the watches is in the error message. So if you see:
```auto
[2017-06-16T18:14:03,422][ERROR][o.e.x.m.e.l.LocalExporter] failed to get monitoring watch [8ix4VdBmSJOml0PwEdjt8w_elasticsearch_cluster_status]
java.lang.IllegalStateException: watch store not started
```
You could try running:
```auto
POST /_xpack/watcher/watch/8ix4VdBmSJOml0PwEdjt8w_elasticsearch_cluster_status/_execute
```
To execute the watch manually and make sure that it is working.
But the bottom line is that these error messages are temporary and can be ignored. It could be happening when the cluster is in recovery due to a node rejoining the cluster. Monitoring will keep trying to do its job to add the watches to the cluster, but it'll fail because the cluster isn't ready. The watches will be added after the cluster recovers because the Monitoring plugin will keep re-trying.
---
### Author:  [@anitakv](https://discuss.elastic.co/u/anitakv)
#### Post date: [August 4, 2017, 2:28pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/16 "2017-08-04T14:28:15Z")
Thanks!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 1, 2017, 2:28pm UTC](https://discuss.elastic.co/t/elasticsearch-errors/89720/17 "2017-09-01T14:28:21Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.