Unless I have missed it, the docs do not seem very clear with regards to how to install watcher on a cluster.
If watcher is installed upon multiple nodes in the cluster, does it generate multiple alerts / actions or is this co-ordinated somehow?
I am a little unsure on whether I need to install the plugin on all of my nodes, or set up a specific (non-master, non-data) watcher node to do the watching (SPOF of course).
Actually, my temptation is to set it up on our (3) master nodes and giving the master nodes the job of monitoring the cluster, which seems broadly appropriate.
Any advice on how to structure this?