How to create _watcher index

Elastic Stack Elasticsearch
1

Hi Team,
Please suggest how can i create/start watcher index? i am getting below error:

i deleted watcher index to resolve an issue and its not working since then, i updated elasticsearch.yml for: watcher.index.rest.direct_access: true

{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[es-master-node][1.1.1.1:9300][cluster:admin/watcher/watch/put]"
}
],
"type": "illegal_state_exception",
"reason": "not started"
},
"status": 500
}

GET .watches/_stats: (doesn't show any failure)

GET _watcher/_stats
{
"error": {
"root_cause": [
{
"type": "index_not_found_exception",
"reason": "no such index",
"index": "_watcher",
"resource.type": "index_or_alias",
"resource.id": "_watcher"
}
],
"type": "index_not_found_exception",
"reason": "no such index",
"index": "_watcher",
"resource.type": "index_or_alias",
"resource.id": "_watcher"
},
"status": 404
}

Thanks & Regards

2

Hey,

have you tried the Start API?

--Alex

3

No, but this is what i got:

[2017-01-24 16:00:27,956][WARN ][rest.suppressed ] /_watcher/_start Params: {}
RemoteTransportException[[es-master-node][1.1.1.1:9300][cluster:admin/watcher/service]]; nested: NullPointerException;
Caused by: java.lang.NullPointerException
at org.elasticsearch.watcher.execution.TriggeredWatchStore.validate(TriggeredWatchStore.java:87)
at org.elasticsearch.watcher.execution.ExecutionService.validate(ExecutionService.java:109)
at org.elasticsearch.watcher.WatcherService.validate(WatcherService.java:97)
at org.elasticsearch.watcher.WatcherLifeCycleService.start(WatcherLifeCycleService.java:97)
at org.elasticsearch.watcher.WatcherLifeCycleService.start(WatcherLifeCycleService.java:64)
at org.elasticsearch.watcher.transport.actions.service.TransportWatcherServiceAction.masterOperation(TransportWatcherServiceAction.java:67)
at org.elasticsearch.watcher.transport.actions.service.TransportWatcherServiceAction.masterOperation(TransportWatcherServiceAction.java:38)
at org.elasticsearch.action.support.master.TransportMasterNodeAction.masterOperation(TransportMasterNodeAction.java:90)
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$3.doRun(TransportMasterNodeAction.java:177)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Tried:

PUT _watcher/_stop
PUT _watcher/_start
GET _watcher/stats:
{
"watcher_state": "stopped",
"watch_count": 0,
"execution_thread_pool": {
"queue_size": 0,
"max_size": 0
},
"manually_stopped": true
}

4

Hey,

which elasticsearch version is this?

--Alex

5

ES Version: 2.3.1

6

Hey,

do you have any unassigned shards of the .triggered_watches index? You can try to delete that index and then restart watcher.

--Alex

7

Delete .triggered_watches-* and .watches_history-* , also deleted their index pattern,

But when try to start, got same error:

PUT _watcher/_start:

{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[es-master-node][1.1.1.1:9300][cluster:admin/watcher/service]"
}
],
"type": "null_pointer_exception",
"reason": null
},
"status": 500
}

8

Can you paste the output of the cat shards API please?

9

That would be a long list, attached might help:

Pasted image1280×54 5.41 KB

10

Hey,

yes, that was actually helpful. No shards are unassigned, thats what I wanted to know.

Can you try the following

  • Ensure that the index template triggered_watches exists
  • Create an index named .triggered_watches
  • Ensure that the index is in an green state
  • Restart watcher again
  • Paste the exception message from the master log if occured

--Alex

11

One more thing, can you share your cluster state? It's going to be big, so it would be nice, if you can put it into some gist or something.

In addition, can you paste the output from

GET _cat/indices/.triggered-watches
12

Is is possible, that your .triggered_watches index is closed?

Can you open it again?

Also note, that this needs to be .triggered_watches with an underscore instead of a dash

13

Yup, resolved. The template was missing.

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.