Is it possible to create a dynamic, rolling interval within a date_histogram?

One of our current monitors, uses a date_histogram to break the last two days worth of data into buckets in hour intervals:

"aggs": {
"histo": {
"date_histogram": {
"field": "@timestamp",
"interval": "1h"
},

I think do a bucket_script on each of these intervals, and compute success rate. I can then do an extended stats bucket to generate the average/std deviation across these buckets to generate a baseline to compare my last bucket to.

The issue I'm having is in the way that ElasticSearch breaks down the intervals. Doing a 1 hour interval doesn't break the buckets into the last 60 minutes, instead, it is based off the buckets generated forward using the interval starting at like 1/1/1970 or something like that. What happens then is, when my alert runs at 1:05 pm, the last "last bucket" really only contains 5 minutes worth of data, which means I'm comparing a much lower volume, where 1 error has a drastic impact on the success rate.

Is there anyway to create dynamic rolling time intervals that are truly the last 60 minutes?

I know you can use offset, but I would need an offset that changes depending on when the script is ran.

I would need something like "offset" : "now - (60 - (now().min))". (Just made up that syntax, but I'm sure you get my point, where now().min returns the minutes past the current hour. So that at 1:05, the offset moves back 55 minutes, and the the last "hour" bucket is actually from 12:05 -> 1:05 giving a true hour's worth of data.

Is this possible with an inline script? Can I use a script to create a parameter/variable and shove that into offset dynamically?

1 Like

Still havn't figured this one out. Anyone able to do something similar?

Hey,

would the offset parameter in the date histogram aggregation work for you?

--Alex

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.