Watcher for percentage drop

Elastic Stack Elasticsearch
1

My use case is a watcher for percentage drop. I have a query from a single field say “submit” and I want to get the percentage drop for the “submit”. Something like c_submit will be the current submit and p_submit is the submit 5 minutes earlier than c_submit. So, say the latest query for “submit” is 10, that will be the value for c_submit and say the query 5 minutes ago for “submit” was 20, that will be the value for p_submit, then I’ll get the percentage.

c_submit = 10
p_submit = 20

Script will be c_submit / p_submit * 100
10 / 20 * 100 = 50%

50% will be the percentage drop, so, if I have my threshold set to < 60, alert should be triggered.

Need help on how I can achieve a working watcher for the above case. Thank you very much in advance.

2

Hey,

you have not mentioned at all, where your problem is. Is it writing the query? Is it writing the condition? Or the action? Maybe you start, where you cannot proceed, and we go from there.

Also, when writing watches, you should check out this blog post, which guides you through the most productive experience writing watches.

--Alex

3

Hi Alex,

Thank you very much for your reply.

So far, here's what I have:

{
"size": 0,
"query": {"bool": {"must": [
{"query_string": {"query": "cte.subcategory:"Report Submitted""}},
{"range": {"@timestamp": {
"from": "now-15m",
"to": "now"
}}}
]}},
"aggs": {"month": {
"date_histogram": {
"field": "@timestamp",
"interval": "month"
},
"aggs": {
"c_submit-data": {
"filter": {"term": {"cte.subcategory.analyzed": "submitted"}},
"aggs": {"c": {"value_count": {"field": "cte.subcategory"}}}
},
"p_submit-data": {
"filter": {"term": {"cte.subcategory.analyzed": "submitted"}},
"aggs": {"c": {"value_count": {"field": "cte.subcategory"}}}
},
"submit-percentage": {"bucket_script": {
"buckets_path": {
"c_submit": "c_submit-data>c",
"p_submit": "p_submit-data>c"
},
"script": "c_submit / p_submit * 100"
}}
}
}}
}

I'm already good with the value for c_submit. What I am trying to get is the value for p_submit which is from the same data but 5 minutes earlier than the data for c_submit.

Hope you can help me on this.

Regards,
Paul

4

I am confused now. Is this a watcher question or is this a pipeline aggregations question?

Also you query looks confusing. You are querying for the last 15 minutes, but then creating a date histogram with an interval of a month. That does not seem to make sense to me.

You could do two things here: First, simply execute two requests, which are similar, except they filter for different time ranges. Then check the percentage in the condition.

Second, execute one request, search for the whole timerange (the last one and the current one), and then use the filters aggregation or a date range aggregation (note the s at the end), to create one bucket for each time range.

Hope this helps.

--Alex

5

Hi Alex,

Sorry for the confusion but this is really a watcher question. Its a watcher that will trigger if the percentage drop is below 60% or 50%. With regards to the date histogram, I’m just testing it out but what I really need is the current count and the count last five minutes ago. I can get the current count but having a hard time getting the last five minutes count on the same query. Can you give me a sample query that will filter from two different time range so I can use it as reference.

Thanks and regards,
Paul

6

Hey Paul,

The easiest solution would be to use a chained input like this

input: {
  chained: {
    "inputs" : [
       "current" : {
           "search" : YOUR_SEARCH_FOR_LAST_5_MINS-GOES-HERE
        },
       "last" : {
           "search" : YOUR_SEARCH_FOR_LAST_5_TO_10_MINS-GOES-HERE
        }
    ]
  }
}

This executes two searches, and allows you to check the two searches in your condition

condition: {
  "script" : {
     "source" : "return (ctx.payload.current.aggregations.foo.bar.anything + 0.0 / ctx.payload.last.aggregations.foo.bar.anything) < 0.6"
  }
}

This is untested pseudocode in both samples, the path foo.bar.anything is of course made up you but should give you a first indication, what I meant.

You can read more about the chain input in the docs.

--Alex

1 Like
7

Thank you very much Alex.

I will try your recommended approach and will let you know how it goes.

Regards,
Paul

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.