# X-Pack Watches not initializing properly

**URL:** <https://discuss.elastic.co/t/x-pack-watches-not-initializing-properly/90116>\
**Category:** Elasticsearch\
**Created:** [June 20, 2017, 2:44pm UTC](https://discuss.elastic.co/t/x-pack-watches-not-initializing-properly/90116 "2017-06-20T14:44:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![bt\_scotth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bt_scotth/32/19313_2.png) [@bt\_scotth](https://discuss.elastic.co/u/bt_scotth)\
**Post date:** [June 20, 2017, 2:44pm UTC](https://discuss.elastic.co/t/x-pack-watches-not-initializing-properly/90116/1 "2017-06-20T14:44:16Z")

</div>

I have ES 5.4.1 installed and the X-Pack plugin is also installed. I am trying to use Watcher, but it is not setting itself up properly.

When I call 'POST \_xpack/watcher/\_start' I get this error in the log files:

[2017-06-20T07:15:04,815][WARN][r.suppressed] path: /\_xpack/watcher/\_start, params: {}  
org.elasticsearch.transport.RemoteTransportException: [Node1][10.110.0.10:9300][cluster:admin/xpack/watcher/service]  
Caused by: org.elasticsearch.transport.ActionNotFoundTransportException: No handler for action [cluster:admin/xpack/watcher/service]  
at org.elasticsearch.transport.TcpTransport.handleRequest(TcpTransport.java:1471) [elasticsearch-5.4.1.jar:5.4.1]  
at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1369) [elasticsearch-5.4.1.jar:5.4.1]  
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) [transport-netty4-5.4.1.jar:5.4.1]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-codec-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:413) [netty-codec-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-codec-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) ~[netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:134) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.11.Final.jar:4.1.11.Final]  
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.11.Final.jar:4.1.11.Final]  
at java.lang.Thread.run(Thread.java:745) [?:1.8.0\_121]

And when I send a 'GET .watches' request, I get back:

{  
"error": {  
"root\_cause": [  
{  
"type": "index\_not\_found\_exception",  
"reason": "no such index",  
"index\_uuid": "_na_",  
"resource.type": "index\_or\_alias",  
"[resource.id](http://resource.id)": ".watches",  
"index": ".watches"  
}  
],  
"type": "index\_not\_found\_exception",  
"reason": "no such index",  
"index\_uuid": "_na_",  
"resource.type": "index\_or\_alias",  
"[resource.id](http://resource.id)": ".watches",  
"index": ".watches"  
},  
"status": 404  
}

So it looks like the .watches index is not being setup. Is there something explicit that I have to do in order to get the .watches index setup properly?

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [June 20, 2017, 2:58pm UTC](https://discuss.elastic.co/t/x-pack-watches-not-initializing-properly/90116/2 "2017-06-20T14:58:01Z")

</div>

Hey,

How many nodes do you have? Have you installed x-pack on all nodes? Did you restart after installation?

Can you show the output of

```auto
GET _cat/plugins?v
GET _cat/nodes?v

```

--Alex

---

<div class="post-metadata">

**Author:** ![bt\_scotth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bt_scotth/32/19313_2.png) [@bt\_scotth](https://discuss.elastic.co/u/bt_scotth)\
**Post date:** [June 20, 2017, 3:18pm UTC](https://discuss.elastic.co/t/x-pack-watches-not-initializing-properly/90116/3 "2017-06-20T15:18:59Z")

</div>

We have: 3 Master/Data nodes and 2 Client nodes...  
X-Pack is on all nodes ...  
Restart was done after upgrade from 5.3 ... as well as being restarted recently after setting 'xpack.watcher.enabled: true' in the elasticsearch.yml file.

Here is the output you requested ...

```auto
GET _cat/plugins?v
name component version
FE2 x-pack 5.4.1
Node3 x-pack 5.4.1
FE1 x-pack 5.4.1
Node1 x-pack 5.4.1
Node2 x-pack 5.4.1

GET _cat/nodes?v
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.110.0.26 8 55 1 - - FE2
10.110.0.7 25 54 5 mdi - Node3
10.110.0.25 11 67 4 - - FE1
10.110.0.10 86 77 11 mdi * Node1
10.110.0.11 35 78 9 mdi - Node2
```

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [June 20, 2017, 5:49pm UTC](https://discuss.elastic.co/t/x-pack-watches-not-initializing-properly/90116/4 "2017-06-20T17:49:04Z")

</div>

Hey,

can you share the settings or the cluster state? It looks as if Node1 still has watcher disabled, given the endpoint cannot be found.

--Alex

---

<div class="post-metadata">

**Author:** ![bt\_scotth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bt_scotth/32/19313_2.png) [@bt\_scotth](https://discuss.elastic.co/u/bt_scotth)\
**Post date:** [June 20, 2017, 7:07pm UTC](https://discuss.elastic.co/t/x-pack-watches-not-initializing-properly/90116/5 "2017-06-20T19:07:28Z")

</div>

Ran 'GET \_nodes' and found that Node1 didn't have watcher enabled like you suspected. Turned out the elasticsearch.yml configuration file wasn't updated properly. I updated the configuration and restarted and it looks like the .watches index was created.

Watcher stats now returns this:

```auto
{
  "watcher_state": "started",
  "watch_count": 4,
  "execution_thread_pool": {
    "queue_size": 0,
    "max_size": 56
  },
  "manually_stopped": false
}
```

Thanks for the help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 18, 2017, 7:07pm UTC](https://discuss.elastic.co/t/x-pack-watches-not-initializing-properly/90116/6 "2017-07-18T19:07:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
