Scaling Watcher to a Scalable Framework - versus a hack pile of code

Our organization is trying to create a scalable framework for how any alert built by watcher analysis logic will (a) have a configuration metdata setup for each watcher and (b) have a well-formed output index structure that is suited for exposing all our alerts nicely on a professional business portal in kibana for exploration and dashboard 'heat' analysis for business operations staff.

Watcher that we did so far is a bit of a hack. Spaghetti code at best with things like distribution lists embedded in code done for the alerts, no functional encapsulation of key modules that build watchers.. basically a bunch of watchers all repeating same logic over and over again.. quite a mess.

Has anyone built a framework on watcher for configurating/storing properties for all your watchers and also encapsulating logic in a scalable way ?

Most vendors seem to do a nice job of encapsulating a service ( watcher ) with helper UI admin config aids and standard structures for setting things up in a clean/scalable way like the distribution list topic above... but we see nothing.

Thoughts ? We are concerned really that watcher doesn't do that much in a sophisticated way and that we already have alot of poor formed logic in watcher that doesn't scale well to build more and more alerts in watcher.

Any advice ? Places to look ?


you are right, watches itself do not have any means of reuse now, so you would need to build this on top. Some months ago I hacked a small prototype that created watches from a git repository in order to make it easier to maintain, which also allows you to use some templating and include other files for reuse.

You can check it out at


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.