Watcher, Creating a kibana watcher to check an integer[0-9] value appears in logs in a give period of time

Hi,

I'm creating a kibana watcher to check an integer field has a value runs between 0 and 9.

A field of type Integer, will have values between 0 through 9 on each record.
The business conditions are,

  1. Each value should have at least one record.
  2. Say a value 2, doesn't have any record then watcher should trigger an email.

My question is

  1. Can I have like a function in watcher and call it iteratively for each value between 0 through 9 and check the response for records to trigger the email alert?
  2. Should I have to create 10 watcher for each value?

What is a simplest and best way to solve my requirement?

thanks in advance
Fredrick

I can try to help you out. But first, if you're on a the leading edge of Kibana releases you could try using Alerting instead of Watcher. The Alerting interface is Beta in the latest 7.9.2 release.

But for now, I'll focus on Watcher. First you need to know the query to trigger your watch.

If I have some test data with a field named my_int which only has some values between 0 and 3.
If I want to find docs which DON'T have this field, then in Discover I could put this in the query bar;
not my_int: * (which means find docs where this field doesn't exist)

If I want to find docs where the value is between 1 and 2 (like your 0 through 9) I could do;
my_int> 0 and my_int < 3 or my_int>= 1 and my_int <= 2

Combining those I could use (my_int>= 1 and my_int <= 2) or not my_int : *

But that is in the KQL query language and we can't use that in our watch. So if we use the "Inspect" menu in Kibana we can see the actual query "Request". This is what Kibana is sending to Elasticsearch but It could almost certainly be simplified. You could take your query to the Kibana Dev Tools Console to test it.

"query": {
"bool": {
  "must": [],
  "filter": [
    {
      "bool": {
        "should": [
          {
            "bool": {
              "filter": [
                {
                  "bool": {
                    "should": [
                      {
                        "range": {
                          "my_int": {
                            "gte": 1
                          }
                        }
                      }
                    ],
                    "minimum_should_match": 1
                  }
                },
                {
                  "bool": {
                    "should": [
                      {
                        "range": {
                          "my_int": {
                            "lte": 2
                          }
                        }
                      }
                    ],
                    "minimum_should_match": 1
                  }
                }
              ]
            }
          },
          {
            "bool": {
              "must_not": {
                "bool": {
                  "should": [
                    {
                      "exists": {
                        "field": "my_int"
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              }
            }
          }
        ],
        "minimum_should_match": 1
      }
    },
    {
      "range": {
        "@timestamp": {
          "gte": "2020-10-19T01:01:00.000Z",
          "lte": "2020-10-19T01:01:05.000Z",
          "format": "strict_date_optional_time"
        }
      }
    }
  ],
  "should": [],
  "must_not": []
}

Once you know the query you'll run, just follow the steps in our docs to create the watch. https://www.elastic.co/guide/en/kibana/current/watcher-ui.html#watcher-create-advanced-watch

In that advanced watch, look for the "query" and put your query there.

Let us know if you have more questions.

Lee

Thank you so much Lee.
We are using ver 7.8.0 and no idea when this will be upgraded.

  1. I saw Alerts and Actions Beta version and I tried to have search logic there but not able to have my custom query. Thats why I jumped back to Watcher. If I can make use of my custom query in alerts please guide me.
  2. I was able to have my custom search query and alerts have started triggering using Watcher. But It looks like I have to have more watcher for each condition, which I don't like. I want to have all my BR in single watcher If there is a way.

Here is my Business Requirement. I'm checking 3 fields and triggering email alert
Out of three fields one among them has to be changed and check the logs to trigger email alert.
like below
"Field1" "Field2" "Field3" "Condition"

  1. "AAA".    "XYZ".           0          if No records found then trigger email for 0.
    
  2. "AAA"     "XYZ".           1          if no records found then trigger email for 1.
    
  3. "AAA"     "XYZ".           2          if no records found then trigger email for 2.
    

.
.
9. "AAA" "XYZ". 9 if no records found then trigger email for 9.

like wise I have to check for Field3 from 0 through 9. In this case, I have to write 10 separate watcher for each Field3 value. Also I have 3 different Field1 values so If I calculate for each value 10 watchers and that ends up in 30 watchers. Which I don't like.

My question is,
Can I have all condition in a single Watcher so that I can have one Watcher for each Field1?

Please let me know, If you need more information.

Thanks for you guidance.
Fredrick

@LeeDr, Can you take a look on my query? thanks

Hi Fredrick,

I guess I'm still not clear on how many conditions you combined in your query and into watcher.
Are you needed to send 10 different emails for the different conditions? Or one email if any of the conditions are met?

Can you share what you have so far in your watch (please redact any private data).

Here's one reference in case you haven't seen it;

Thank for checking, @LeeDr.
I had different discussion for the same query and got the response..

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.