Watcher alert when count is zero for a bucket

Elastic Stack Elasticsearch
1

Hi,

I'm trying out watcher and I am having a little trouble in understanding how it works. I'm looking to get alerted when log count for a specific log type becomes zero between the latest timestamp ingested and 5 mins before that. I have 5 logs currently on elasticsearch indexed under logstash-*.

My current script is:

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "prod_log-*"
        ],
        "types": [],
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "must": [
                {
                  "match": {
                    "type": "SystemError"
                  }
                },
                {
                  "match": {
                    "server": "117"
                  }
                },
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-5m",
                      "lte": "now"
                    }
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "eq": 0
      }
    }
  },
  "actions": {
    "send_email": {
      "email": {
        "profile": "standard",
        "to": [
          "gvenkat21@gmail.com"
        ],
        "subject": "117 server down alert",
        "body": {
          "text": "No events generated in the past 5 mins. Could indicate that the server is down. The alert was detected at {{ctx.execution_time}}"
        }
      }
    }
  }
}

But since I have a delay of 15-20 mins before log ingestion, the script keeps generating alert. I need it to look at the latest timestamp of the ingested log( which would be approx 20 mins in the past)

Any help would be appreciated.

2

Hey,

I do not follow your problem exactly. What if you dont use a timestamp range, but you are just returning the latest document that has been indexed and compare that with the execution timestamp of the watch (which is in {{ctx.execution_time}}, you could use a scripted condition to compare those two values and decide if the span between those two dates is too high, then this system has not sent any data.

Would that work better for you? If not, please explain your usecase a bit more detailed. Thank you!

--Alex

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.