Watcher - email on condition that doc_count is less than certain value

Hi,

I have created a watcher that checks various target servers to see if 3 files are present at each target location.
I have a number of locations to check.
As the email alert must alert at a specific time, I have grouped these checks into 1 watcher.
I only want an email to be sent if there are less than 3 files at a particular target location.

Here is my (sanitised) query:

GET /auditbeat-*/_search/
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "file.ctime": {
              "gte": "now-6d/d",
              "lte": "now/d"
            }
          }
        },
                {
                  "bool": {
                    "minimum_should_match": 1,
                    "should": [
                      {
                        "query_string": {
                          "query": "(host.hostname: host123) AND (event.action: created) AND (file.path: *regex1*)"
                        }
                      },
                      {
                        "query_string": {
                          "query": "(host.hostname: host123) AND (event.action: created) AND (file.path: *regex2*)"
                        }
                      },
                      {
                        "query_string": {
                          "query": "(host.hostname: host123) AND (event.action: created) AND (file.path: *regex3*)"
                        }
                      }
                    ]
                  }
                }
              ]
            }
          },
          "sort": {
            "file.path": "desc"
          },
          "aggs": {
            "filegroup": {
              "filters": {
                "filters": [
                  {
					"query_string": {
					  "query": "(host.hostname: host123) AND (event.action: created) AND (file.path: *regex1*)"
					}
				  },
				  {
					"query_string": {
					  "query": "(host.hostname: host123) AND (event.action: created) AND (file.path: *regex2*)"
					}
				  },
				  {
					"query_string": {
					  "query": "(host.hostname: host123) AND (event.action: created) AND (file.path: *regex3*)"
					}
				  }
                ]
              },
              "aggs": {
                "host_name": {
                  "terms": {
                    "field": "host.hostname",
                    "size": 1
                  }
                },
                "server_tag": {
                  "terms": {
                    "field": "tags",
                    "size": 1
                  }
                },
                "filename": {
                  "terms": {
                    "field": "file.path",
                    "size": 100,
                    "order": {
                      "_key": "desc"
                    }
                  }
                }
              }
            }
          }
}

and my (sanitised) results look like this

"aggregations" : {
    "filegroup" : {
      "buckets" : [
        {
          "doc_count" : 3,
          "filename" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : """myfilename1.txt""",
                "doc_count" : 1
              },
              {
                "key" : """myfilename2.txt""",
                "doc_count" : 1
              },
              {
                "key" : """myfilename3.txt""",
                "doc_count" : 1
              }
            ]
          },
          "server_tag" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 3,
            "buckets" : [
              {
                "key" : "MyServerTag",
                "doc_count" : 3
              }
            ]
          },
          "host_name" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "myserverhostname",
                "doc_count" : 3
              }
            ]
          }
        },```

I would like an email to alert if any of the filters returns less than 3 files. In the email, I will provide details to the user of the hostname and server tag. They do not require an alert if the correct number of files exist.

Can I use filegroup.buckets.doc_count in a condition within the email action to achieve this? I have tried so many different ways, but getting null or ctx error when simulating an alert.

My watcher email syntax looks like this (extract):

"actions": {
    "email_1": {
      "condition": {
        "compare" : { "ctx.payload.aggregations.filegroup.doc_count" : { "lt" : 3 } }
      },
      "foreach": "ctx.payload.aggregations.filegroup.buckets",
      "max_iterations": 500,
      "email": {
        "profile": "standard",
        "priority": "high",```

I have tried a few variations on the condition within the email action.

Any advice would be appreciated, perhaps I have the approach completely wrong.

Thank you

I have not read the whole query but the condition seems to be

"compare" : { "ctx.payload.aggregations.filegroup.buckets.doc_count" : { "lt" : 3 }
1 Like

Hi Tomo, thank you for that - I think I’ve tried that or something very similar, but I will double check to be 100% sure!

FYI the doc_count returned 'null'

I am trying with the array_compare now on the email action, but the alert is emailing for each of the queries, rather than just the one that fails.

How about

"compare" : { "ctx.payload.aggregations.filegroup.buckets[0].doc_count" : { "lt" : 3 }

?