Watcher does not get triggered properly

[mhmtsvr (Mehmet)]

Hi,

I have a watcher notifying the slack channel once elastic receives any logs regarding the query I set. It usually triggers, but I sometimes observe that the watcher does not get triggered even though logs are available in the elastic. Is there something wrong with the watcher configuration or the watcher itself?

{
  "trigger": {
    "schedule": {
      "interval": "30s"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "filebeat-*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "query": {
            "bool": {
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-30s",
                      "lte": "now"
                    }
                  }
                }
              ],
              "must": [
                {
                  "regexp": {
                    "fields.auditlog": "<app-name>"
                  }
                },
                {
                  "regexp": {
                    "request.path": "path/.*"
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 1
      }
    }
  },
  "actions": {
    "notify-slack": {
      "throttle_period_in_millis": 30000,
      "slack": {
        "message": {
          "from": "Monitoring App",
          "to": [
            "<#slack-channel-name>"
          ],
              "text": "<text here>"
        }
      }
    }
  }
}

As it can be seen, the watcher is able to get triggered. However, I try to trigger it after 2 mins, it happens nothing. Like I mentioned, the logs are available in elastic.

Any leads on this issue will be highly appreciated. Thanks!