Math operation with aggregated fields

Hello Folks,

A newbie (to watchers at least) is here :slight_smile:

I'm trying to create a watcher for disk_space usage
we have multiple hosts send data to same index pattern, so I was planning to aggregate hits with hostname and get required fields in buckets inside aggregation as _source fields

The Math calculation I need is very simple.

"system.fsstat.total_size.used/system.fsstat.total_size.total > ctx.metadata.thresholdPercent"

Below is my in progress watcher and I know for certain that, this condition does not work like this

{
  "trigger": {
    "schedule": {
      "interval": "30m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "metricbeat-BlaBla-*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "query": {
            "bool": {
              "must": [
                {
                  "query_string": {
                    "query": "(system.fsstat.total_size.total : *)"
                  }
                },
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-30m"
                    }
                  }
                }
              ]
            }
          },
          "_source": [
            "host.name",
            "system.fsstat.total_size.total",
            "system.fsstat.total_size.used",
            "system.fsstat.total_size.free"
          ],
          "sort": [
            {
              "@timestamp": {
                "order": "desc"
              }
            }
          ],
          "aggs": {
            "hostname": {
              "terms": {
                "field": "host.name"
              },
              "aggs": {
                "recent_diskspace_used": {
                  "top_hits": {
                    "sort": [
                      {
                        "@timestamp": {
                          "order": "desc"
                        }
                      }
                    ],
                    "_source": {
                      "includes": [
                        "system.fsstat.total_size.total",
                        "system.fsstat.total_size.used"
                      ]
                    },
                    "size": 1
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "script" :{
        "source": "return ctx.payload.aggregations.hostname.buckets.recent_diskspace_used.hits.hits[0]._source.system.fsstat.total_size.used/ctx.payload.aggregations.hostname.buckets.recent_diskspace_used.hits.hits[0]._source.system.fsstat.total_size.total > ctx.metadata.thresholdPercent",
        "lang": "painless"
      }
  },
  "actions": {
    "my-logging-action": {
      ................................
    }
  },
  "metadata": {
    "thresholdPercent": 0.5
  }
}

And if we check the aggregations they look like this

            "aggregations":{
               "hostname":{
                  "doc_count_error_upper_bound":0,
                  "sum_other_doc_count":0,
                  "buckets":[
                     {
                        "doc_count":30,
                        "recent_diskspace_used":{
                           "hits":{
                              "hits":[
                                 {
                                    "_index":"metricbeat-BlaBla-000005",
                                    "_type":"_doc",
                                    "_source":{
                                       "system":{
                                          "fsstat":{
                                             "total_size":{
                                                "total":107372081152,
                                                "used":48694804480
                                             }
                                          }
                                       }
                                    },
                                    "_id":"gVimMnIBG9oIUupGvXgq",
                                    "sort":[
                                       1589987687069
                                    ],
                                    "_score":null
                                 }
                              ],
                              "total":30,
                              "max_score":null
                           }
                        },
                        "key":"hostname_one"
                     },
                     {
                        "doc_count":30,
                        "recent_diskspace_used":{
                           "hits":{
                              "hits":[
                                 {
                                    "_index":"metricbeat-BlaBla-000005",
                                    "_type":"_doc",
                                    "_source":{
                                       "system":{
                                          "fsstat":{
                                             "total_size":{
                                                "total":107372081152,
                                                "used":57353330688
                                             }
                                          }
                                       }
                                    },
                                    "_id":"3AWmMnIBzr28qyZPNQYk",
                                    "sort":[
                                       1589987651869
                                    ],
                                    "_score":null
                                 }
                              ],
                              "total":30,
                              "max_score":null
                           }
                        },
                        "key":"hostname_two"
                     }
                  ]
               }
            }

In the end, what I need to do is.

If the math calculation result is false (per hostname), I need to trigger action.
So need to evaluate hits for each hostname aggregation field values...

I know this is not that complicated.
However, I was lost in Elastic documentation pages.....
Any help is much appreciated...

Hey @spinscale,

Sorry to spam you with the tag.
Will you be able to shed some light on this?

Thanks in advance.

Hey,

so a script condition is what you are after.

return ctx.payload.aggregations.buckets.stream().anyMatch(b -> { size = b.recent_diskspace.used.hits.hits[0]._source.system.fsstat.total_size; return size.used/size.total > THRESHOLD)}

This above returns true if any of the buckets matches the inside condition. I hope that helps as a start.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.