[6.1.1] Machine Learning Job: "Rolling" Datafeed Period

Elastic Stack Elasticsearch
1

Hello!

I'm testing some ML Jobs with daily bucket span over my Elasticsearch data for different datafeed periods and I've noticed different anomaly points as I changed the datafeed start time. For example, the job below is configured to query all data (Start at beginning of data option) until now (No end time option):

11
Screen Shot 2018-02-06 at 17.42.11.png696×271 13.9 KB

This second example's datafeed started at the end of January (24/01) until now:

47
Screen Shot 2018-02-06 at 17.42.47.png695×268 16.2 KB

Probably the first one wasn't an anomaly because of the past metric value (since datafeed is querying all data), but it should have been an anomaly compared to the last 10 days.

Is there a way to configure the start/end time of my ML Job datafeed in order to consider only "Last x days" in analysis (something like a "rolling" period: now-10d)?

Thanks!

2

Hello Gustavo,

Yes, you will obviously get different results depending on when you start your analysis and the characteristics of the data during that period of time.

ML is designed to be constantly evaluating your data, understanding and modeling its behaviors over all of the time it has seen your data. The main benefit is near-real-time alerting without having to set a static threshold alert.

If you're only interested in a visual analysis over a window of time, I'd encourage you to look at the features of timelion (https://www.elastic.co/blog/timelion-tutorial-from-zero-to-hero)

Hope that helps!