# Generate Alert for entries not present in the log for a given time window

**URL:** https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874
**Category:** Kibana
**Tags:** elastic-stack-alerting, kql-kibana-query-language
**Created:** [August 21, 2020, 8:12am UTC](https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874 "2020-08-21T08:12:42Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![akhettar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhettar/32/74293_2.png) [@akhettar](https://discuss.elastic.co/u/akhettar)
#### Post date: [August 21, 2020, 8:12am UTC](https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874/1 "2020-08-21T08:12:42Z")

</div>

Hello there,

I am hoping if somebody can help me write monitor's query to generate an alert to be pushed to a slack channel. I am using ELK stack v 7.4.2.

Here is the payload of the message:

```auto
{
   "metaData":{
      "timestamp":"2020-08-21T08:04:24.523Z",
      "appVersion":"0.0.1-SNAPSHOT",
      "appName":"cr-inventory-files-uploader",
      "logger":"http-nio-8080-exec-5",
      "priority":"INFO",
      "envName":"dev",
      "envHost":"localhost",
      "tracePoint":"START"
   },
   "payload":{
      "class":"com.vfc.mkpl.zacr.inventoryfilesuploader.controllers.FileUploadController:47",
      "message":"Received new inventory file: INVENTORY-PRICE_S307_20200716183709.csv",
      "customData":{

      },
      "exception":""
   },
   "context":{
      "correlationRootId":"edbec86e-210f-4529-abd0-e1cdc28266e6",
      "customData":{
         "file-size":74,
         "file-name":"INVENTORY-PRICE_S307_20200716183709.csv",
         "store-id":"S307",
         "user-agent":"AmazonAPIGateway_3tlidsdlq9"
      },
      "correlationId":"23a2a574-1151-4214-85da-f680664c6540"
   }
}

```

Basically the Monitor's query needs to check the presence of the above log for a given time window (1minute) and if not present it raises an alert. See below sample query I wrote but it is matching in all cases even if the log entry is present. The **must** clause works but **must\_not** is not working. Have I missed anything?

thanks in advance

```auto
{
    "size": 0,
    "query": {
        "bool": {
            "must_not": [
               
                {
                    "match_phrase": {
                        "customData.store-id": {
                            "query": "S307",
                            "slop": 0,
                            "zero_terms_query": "NONE",
                            "boost": 1
                        }
                    }
                },
                {
                    "match_phrase": {
                        "metaData.tracePoint": {
                            "query": "START",
                            "slop": 0,
                            "zero_terms_query": "NONE",
                            "boost": 1
                        }
                    }
                },
                {
                    "range": {
                        "@timestamp": {
                            "from": "now-1m",
                            "to": "now",
                            "include_lower": true,
                            "include_upper": true,
                            "format": "strict_date_optional_time",
                            "boost": 1
                        }
                    }
                }
            ],
            "adjust_pure_negative": true,
            "boost": 1
        }
    }
}

```

 ![Screenshot 2020-08-21 at 10.04.52](https://us1.discourse-cdn.com/elastic/original/3X/4/c/4c91e12aafa9096b36a7a92d594f26f17ec60596.png)

---

<div class="post-metadata">

### Author: ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)
#### Post date: [August 24, 2020, 5:03pm UTC](https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874/2 "2020-08-24T17:03:47Z")

</div>

You probably need to set `minimum_should_match` to 3, if you want all clauses to match. [https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html)

---

<div class="post-metadata">

### Author: ![akhettar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhettar/32/74293_2.png) [@akhettar](https://discuss.elastic.co/u/akhettar)
#### Post date: [August 27, 2020, 1:58pm UTC](https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874/3 "2020-08-27T13:58:14Z")

</div>

> [@wylie](#):
>
> `minimum_should_match`

Many thanks Wylie, I tried that but it is not working. Alerts are still being triggered despite the fact there are entries in the logs

---

<div class="post-metadata">

### Author: ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)
#### Post date: [August 27, 2020, 4:15pm UTC](https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874/4 "2020-08-27T16:15:57Z")

</div>

Reading your query again, it seems like you are excluding all documents from the last 1 minute, but your query will still match older documents. I think you want a query like this:

```auto
{
  "size": 0,
  "query": {
    "bool": {
      "must": [{
        "range": {
          "@timestamp": {
            "from": "now-1m",
            "to": "now",
            "include_lower": true,
            "include_upper": true,
            "format": "strict_date_optional_time",
            "boost": 1
          }
        }
      }],
      "must_not": [
        {
          "match_phrase": {
            "customData.store-id": {
              "query": "S307",
              "slop": 0,
              "zero_terms_query": "NONE",
              "boost": 1
            }
          }
        },
        {
          "match_phrase": {
            "metaData.tracePoint": {
              "query": "START",
              "slop": 0,
              "zero_terms_query": "NONE",
              "boost": 1
            }
          }
        },
      ],
      "minimum_should_match": 3,
      "adjust_pure_negative": true,
      "boost": 1
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![akhettar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhettar/32/74293_2.png) [@akhettar](https://discuss.elastic.co/u/akhettar)
#### Post date: [August 31, 2020, 11:50am UTC](https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874/5 "2020-08-31T11:50:44Z")

</div>

Thanks again Wylie for your suggestion. I did try this but it is not matching, I do't know really what I am missing. I am thinking of rewritting the query to look for something in the log rather check if it's not there. I don't have such thing in the log right now, it has to be generated by the application hence I started with `must-not` query.

---

<div class="post-metadata">

### Author: ![wylie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wylie/32/81794_2.png) [@wylie](https://discuss.elastic.co/u/wylie)
#### Post date: [August 31, 2020, 2:03pm UTC](https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874/6 "2020-08-31T14:03:57Z")

</div>

I agree that it would be easier to create positive matches, since you could easily see when it's successful.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 28, 2020, 2:04pm UTC](https://discuss.elastic.co/t/generate-alert-for-entries-not-present-in-the-log-for-a-given-time-window/245874/7 "2020-09-28T14:04:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
