Alert and action -- Less than condition is not working

[msszafar (zafar iqbal)]

Hi All,

Thank you for this great support where we get reply in few minutes. We're very grateful to all of you guys.

We're using latest version of ELK. In alert and action > Log Threshold, less than feature doesn't work as highlighted in the screenshot

We're intended to get alert when a logsource stop sending logs, here if we change it with more than X number of logs entries then it works and trigger an alert but less than doesn't work.

Here are two observations:
1st observation, if logsource i.e host.name wp-pri stop sending logs then in realtime wp-pri will not be appearing in the realtime logs that's why my condition becomes false. It make sense.

But here I'm doing it like wp-pri hostname is appreaing in the real time logs. I know in last 30 minutes this hostname doesn't have more than 2000 json documents, it should have trigger an alert now. When I change it like if more than 1000 logs occur then it trigger an alert successfully.

Please help me what's happening here and how can I accomplish my goal.

Thank you.

[msszafar (zafar iqbal)]

Hi Community,

Can anyone please reply on this. Intention is to get alert if logsource stops sending logs

[stephenb (Stephen Brown)]

What version of Kibana and what type of alert are you creating?
Is that a Log Threshold Alert?

Also what action type are you using?

One thing I notice you are filtering by host.name and group by host.name ...

What happens if you change group by to Nothing

[msszafar (zafar iqbal)]

Sir,

Kibana version is 7.9.0 and I'm creating log threshold alert.

Alert Type: Log Threshold
Action Type: Index

P.S: There is no problem with index. I have verified that alert json document save on the index.

[stephenb (Stephen Brown)]

I just wrote this is the less than and it worked

And the Alert Fired

  {
    "_index" : "alerts",
    "_type" : "_doc",
    "_id" : "XwE_TXQBsa3bYJU-EdKd",
    "_score" : 1.0,
    "_source" : {
      "context_message" : "",
      "alert_id" : "8315ad77-4eb1-4dca-b112-fc2fbb97a819",
      "alert_name" : "test-log",
      "alert_instance_id" : "*",
      "@timestamp" : "2020-09-02T05:16:53.020Z"
    }
  },

[msszafar (zafar iqbal)]

@stephenb

One thing I notice you are filtering by host.name and group by host.name ...

What happens if you change group by to Nothing

Sir, it didn't effect, now I have group by Nothing

I don't think group by cause any effect because in kibana visualization, we filter any field and then we can group by on that field too.

Here group by is working ok. I could have seen all logsource names in host.name but now it's * when after making changes on group by to Nothing.

Thank you

[stephenb (Stephen Brown)]

Did you go to Discover set time range to 30 mins and filter on the host.name : "wp-pri" and type : "auditd" you get a count of less than 2000 if so the alert should fire or perhaps wrong in the logic.

Curious where that type field came from?

[msszafar (zafar iqbal)]

Sir, I had made it sure before creating rule. I'm sorry, don't know why it doesn't work for me.

[stephenb (Stephen Brown)]

Hmmm there is something interesting going on. It is late where I am at, I will need to take a look tomorrow and perhaps ask for some guidance.

The logic is not working as I understand it... we are trying to mix positive and negative logic.

It appears that perhaps since there are no / 0 documents that match the type and host.name conditions that those conditions are then false and since all the conditions are ANDed that the overall condition is false and thus the alert does not fire.

Your discover showed 0 documents matching... so I think that is what is happening.

I think if there were a couple documents that matched that perhaps the alert would fire.

I will do my best to check, or see if I can get someone else to take a look.

[msszafar (zafar iqbal)]

Thank you sir for you reply and great support!

I have created another connector and assigned that connector to this alert. It worked.

[stephenb (Stephen Brown)]

Hi @msszafar

Thanks for the response and issues ...

A couple things...

1. I did verify that if you filter on a term like host.name IS wp-pri and that filter results in 0 Documents you can not use GROUP BY host.name you can not group by the field that returns 0 results.

2. Can you please post / show your current / working / solution please we are interested.

3. The team actually created an issue related to your request if you want to take a look

[Logs UI] [Alerting] Handle the "no data" case reliably