Elasticsearch Watcher Stops processing after recovering from full disk

Elastic Stack Elasticsearch
1

3 times now I've managed to fill up my disks and after recovering the watcher service fails to work reliably. I end up having to restart each data node then it works fine. This isn't just my watches, but the system watches as well. I've tried stopping the service with
POST _xpack/watcher/_stop
followed by a start
POST _xpack/watcher/_start

it starts erroring right away on the console.
[2018-09-04T22:55:07,503][WARN ][o.e.x.w.e.ExecutionService] [myhost] failed to execute watch [322e52YFQ16I7v2vd9B8kQ_logstash_version_mismatch]
I've not seen any reason for it.
This was just one example.

2

This can be deleted,

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.