Alert if index has 0 documents in last X minutes (Rules and connectors or Detection Rules)

Elastic Stack Kibana
1

Hello everyone,

I'm using kibana version 7.17.2
I work with rule and connector in order to receive an alert if I don't receive any new document in one of my index for the last X minutes.
Select index:

INDEX index*
WHEN count()
GROUPED OVER top 1000 my_field

define the condition

IS BELOW 1
FOR THE LAST 2 days

But it seems like the alert is never triggered even if the condition is met.
Everything works fine when the condition is "above" a value.

I saw this post with a similar issue but I think the version used is below mine.
And when looking at the code for version 7.17.2 I don't see modification made by PR #105626.

Am I doing something wrong?
Or is there an easier way to check if 0 documents are indexed for the last 2 days?

Thank you for your help!

2

I checked the 7.17.2 code, and it looks like the fix you mentioned is applied there:

Previously, the check was if (!value), which also matched the value 0, so we were not considering "0 matches" cases as an alertable case - we debug log a message and continue to the next group to check. We should be considering those 0's now.

The only thing I can think of is whether your index pattern index* might be too broad (with the wildcard at the end), that it's finding more documents than you are expecting.

3

Thanks for your answer!
woops sorry my bad I didn't saw that part of code....

I don't think this is an error with the wildcard because with the preview when writing the rule I see 3 value for my_field which is what I expected, let's say:

  • value 1
  • value 2
  • value 3

value 1 and value 2 are respectively 200 and 42
but for value 3 the preview give me 0 as a result.
I'm then expecting an alert to be triggered for value 3 but that's not case....

Maybe one think I didn't mention previously is that my rule is scheduled to be run once per day.

Did I misunderstand something?
Or did I do something wrong?

Thanks for your help

4

For your rule, what is the Notify parameter set to?

  • Every time alert is active
  • Only on status change
  • On a custom action interval

The default is the second, "Only on status change", which means the rule will only notify when the alert becomes active; if it stays active (does not recover), it won't notify any more. Once the rule recovers, and then goes active again, it will again notify once.

The first, "Every time alert is active", will notify every time the alert is active.

5

It was the default one "Only on status change"
But the thing is the rule never become active and then changing it to "Every Time alert is active" does not change anything.

Then I'm wondering if this works with the "GROUPED OVER" request? Because if for example no document are indexed for value3, how the rules know that this value3 existed before?

6

Oh, yeah, I think you're right. If you're grouping, we use an agg, and there won't be an agg entry if there were no hits on.

In general it's hard to search on data that's not there :slight_smile:

Let me ask around.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.