Spike Detection in a Time-Series

Hi there,

I am trying to build a watch which alerts on spikes (both up or down) over a time series of a specific field. To illustrate this with an example if my values of a field say cpu is [20,23,24,25,50,52,18,19,20,90] , I want to to be notified when cpu hits 50 for upward spike and again at 18 for downward spike.

I seems to hit a roadblock in defining an appropriate watcher query for this as the normal aggregation functions is not helping. It may so happen that I am ignorant on how to create a proper query DSL on elasticsearch for this.

What could I do to fix this ?

1 Like

+1, I am interested in this as well

Even I am interested. This is one of general need for alerts. I think watcher should come up with it.

Apologies for the long gap in reply here. In Elasticsearch 2.0, we added a new type of aggregations, called pipeline aggregations, which make it easy to do various types of math on the output of existing aggregations.

Zach wrote a great 3-part blog post about how to build a statistical anomaly detector using pipeline aggregations and Watcher, our alerting and automation product.

1 Like

Just putting this here for future viewers to see that advanced machine learning-based anomaly detection was introduced in v5.4 and went GA in v5.5: