Spike Detection in a Time-Series

Elastic Stack Elasticsearch
1

Hi there,

I am trying to build a watch which alerts on spikes (both up or down) over a time series of a specific field. To illustrate this with an example if my values of a field say cpu is [20,23,24,25,50,52,18,19,20,90] , I want to to be notified when cpu hits 50 for upward spike and again at 18 for downward spike.

I seems to hit a roadblock in defining an appropriate watcher query for this as the normal aggregation functions is not helping. It may so happen that I am ignorant on how to create a proper query DSL on elasticsearch for this.

What could I do to fix this ?

1 Like
2

+1, I am interested in this as well

3

Even I am interested. This is one of general need for alerts. I think watcher should come up with it.

4

Apologies for the long gap in reply here. In Elasticsearch 2.0, we added a new type of aggregations, called pipeline aggregations, which make it easy to do various types of math on the output of existing aggregations.

Zach wrote a great 3-part blog post about how to build a statistical anomaly detector using pipeline aggregations and Watcher, our alerting and automation product.



1 Like
6

Just putting this here for future viewers to see that advanced machine learning-based anomaly detection was introduced in v5.4 and went GA in v5.5:

https://www.elastic.co/products/x-pack/machine-learning