I am trying to build a watch which alerts on spikes (both up or down) over a time series of a specific field. To illustrate this with an example if my values of a field say cpu is [20,23,24,25,50,52,18,19,20,90] , I want to to be notified when cpu hits 50 for upward spike and again at 18 for downward spike.
I seems to hit a roadblock in defining an appropriate watcher query for this as the normal aggregation functions is not helping. It may so happen that I am ignorant on how to create a proper query DSL on elasticsearch for this.
Apologies for the long gap in reply here. In Elasticsearch 2.0, we added a new type of aggregations, called pipeline aggregations, which make it easy to do various types of math on the output of existing aggregations.
Zach wrote a great 3-part blog post about how to build a statistical anomaly detector using pipeline aggregations and Watcher, our alerting and automation product.