How to create _watcher index


(piyush) #1

Hi Team,
Please suggest how can i create/start watcher index? i am getting below error:

i deleted watcher index to resolve an issue and its not working since then, i updated elasticsearch.yml for: watcher.index.rest.direct_access: true

{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[es-master-node][1.1.1.1:9300][cluster:admin/watcher/watch/put]"
}
],
"type": "illegal_state_exception",
"reason": "not started"
},
"status": 500
}

GET .watches/_stats: (doesn't show any failure)

GET _watcher/_stats
{
"error": {
"root_cause": [
{
"type": "index_not_found_exception",
"reason": "no such index",
"index": "_watcher",
"resource.type": "index_or_alias",
"resource.id": "_watcher"
}
],
"type": "index_not_found_exception",
"reason": "no such index",
"index": "_watcher",
"resource.type": "index_or_alias",
"resource.id": "_watcher"
},
"status": 404
}

Thanks & Regards


(Alexander Reelsen) #2

Hey,

have you tried the Start API?

--Alex


(piyush) #3

No, but this is what i got:

[2017-01-24 16:00:27,956][WARN ][rest.suppressed ] /_watcher/_start Params: {}
RemoteTransportException[[es-master-node][1.1.1.1:9300][cluster:admin/watcher/service]]; nested: NullPointerException;
Caused by: java.lang.NullPointerException
at org.elasticsearch.watcher.execution.TriggeredWatchStore.validate(TriggeredWatchStore.java:87)
at org.elasticsearch.watcher.execution.ExecutionService.validate(ExecutionService.java:109)
at org.elasticsearch.watcher.WatcherService.validate(WatcherService.java:97)
at org.elasticsearch.watcher.WatcherLifeCycleService.start(WatcherLifeCycleService.java:97)
at org.elasticsearch.watcher.WatcherLifeCycleService.start(WatcherLifeCycleService.java:64)
at org.elasticsearch.watcher.transport.actions.service.TransportWatcherServiceAction.masterOperation(TransportWatcherServiceAction.java:67)
at org.elasticsearch.watcher.transport.actions.service.TransportWatcherServiceAction.masterOperation(TransportWatcherServiceAction.java:38)
at org.elasticsearch.action.support.master.TransportMasterNodeAction.masterOperation(TransportMasterNodeAction.java:90)
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$3.doRun(TransportMasterNodeAction.java:177)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Tried:

PUT _watcher/_stop
PUT _watcher/_start
GET _watcher/stats:
{
"watcher_state": "stopped",
"watch_count": 0,
"execution_thread_pool": {
"queue_size": 0,
"max_size": 0
},
"manually_stopped": true
}


(Alexander Reelsen) #4

Hey,

which elasticsearch version is this?

--Alex


(piyush) #5

ES Version: 2.3.1


(Alexander Reelsen) #6

Hey,

do you have any unassigned shards of the .triggered_watches index? You can try to delete that index and then restart watcher.

--Alex


(piyush) #7

Delete .triggered_watches-* and .watches_history-* , also deleted their index pattern,

But when try to start, got same error:

PUT _watcher/_start:

{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[es-master-node][1.1.1.1:9300][cluster:admin/watcher/service]"
}
],
"type": "null_pointer_exception",
"reason": null
},
"status": 500
}


(Alexander Reelsen) #8

Can you paste the output of the cat shards API please?


(piyush) #9

That would be a long list, attached might help:


(Alexander Reelsen) #10

Hey,

yes, that was actually helpful. No shards are unassigned, thats what I wanted to know.

Can you try the following

  • Ensure that the index template triggered_watches exists
  • Create an index named .triggered_watches
  • Ensure that the index is in an green state
  • Restart watcher again
  • Paste the exception message from the master log if occured

--Alex


(Alexander Reelsen) #11

One more thing, can you share your cluster state? It's going to be big, so it would be nice, if you can put it into some gist or something.

In addition, can you paste the output from

GET _cat/indices/.triggered-watches

(Alexander Reelsen) #12

Is is possible, that your .triggered_watches index is closed?

Can you open it again?

Also note, that this needs to be .triggered_watches with an underscore instead of a dash


(piyush) #13

Yup, resolved. The template was missing.


(system) #14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.