Using script with date_histogram offset

Elastic Stack Elasticsearch
1

I create a watcher running each 10m, and aggregate result with date_histogram interval 15m.
Now I wanna set offset of aggregation based on the time it executes, how should I config with the watcher (version 5.4).
Here is my watcher configuration and offset is the description which expected.

{
  "trigger":{ "schedule":{ "interval":"10m" } },
  "input":{
    "search":{
      "request":{
        "search_type":"query_then_fetch",
        "indices":["my-index-of-log-*"],
        "types":[],
        "body":{
          "size":0,
          "query":{
            "bool":{
              "must":[
                { "range":{ "log_date":{ "gte":"now-6h", "lte":"now" } } },
                { "term":{ "level":"error" } }
              ]
            }
          },
          "aggs":{
            "log_date":{
              "date_histogram":{
                "field":"log_date",
                "interval":"15m",
                "offset":"something values `(now.date.minuteOfHour % 15) m`"
              }
            }
          }
        }
      }
    }
  },
  "condition":{ },
  "actions":{ }
}
2

Nothing comes to mind on top of my head to solve this.

Can you explain why you need dynamic offset per execution (which makes previous runs harder to compare), I'm interested in the use-case here.

--Alex

3

My alert condition is :

  1. error log appeared more than 20 counts in last 15 minutes
  2. AND notification only trigged up to 5 time in 6 hour

The second rule is designed to prevent too many notifications to disturb users.
That is why I need the result aggregated in date_histogram.

My watcher condition is not set, and the action is just post the result to a webhook. The webhook will deside whather count of result buckets satisfing the 1st rule reaches the max time set(that is 5, in the 2nd ruld).

Now comes my concern. The watch runs once 10 minutes. If 10 errors occur in 00:10~00:15 and 10 errors occur in 00:15~00:20 that may mean neither 00:00~00:15 or 00:15~00:30 bucket satifies the 1st rule with fixed offset, and mean 00:05~00:20 bucket satisifies with dynamic offset.

I think this situation is better to notify, so I am trying setting dynamic offset.
However, I have no idea if this is a good point.

Hope I have described my problem clearly.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.