Need help with creating Kibana Alert

Hello everyone,
I am new to the Elasticstack in general and I have encountered a problem with creating an Alert using the Elasticsearch Query. I am using the index from metricbeat that indexes various processes of a host. From there I want the number of matches in the threshold to be below 1 so that the action is triggered whenever there is 0 match (meaning that the process is not working).
But this is not the case in my testing as when I stopped the host server to test the alert, the alert status would just change from active to ok with no instance of the alert to be found.
Could someone please help me with this problem? Thank you in advance!

Hi @Long_Nguyen Welcome to the community.

Yes that is interesting... I got the same result.

I will check internally if that is a bug or the designed behavior.

In the meantime I did find a hack / work around if you are interested...

1 Like

Hi @stephenb ,

Thank you so much for the reply. Yes, I am interested in the work around. Could you please show me? I would appreciate it a lot! :slight_smile:

Ok as I said it is a Hack or maybe we should just call it creative .. :slight_smile:

So we will use a Log Threshold alert.

To do that we are going to treat your metricbeat-* data as log entries.

To do that you we will add metricbeat-* as a log source under the logs app.

Add metricbeat-* as a log source and save it.

Now we can use Logs Threshold alert

Create an alert with the conditions you want use Add Condition to add additional conditions
You can even group by host.name so if the process is working on 1 host but not the other it will figure that out and only alert on the ones that it is missing on (Kinda Cool right?)

Give that a try let me know....

1 Like

Hi @stephenb ,

Thank you for the reply. I just tried the hack this morning. However to my surprise, I found what could be a bug on my Kibana (version 7.12.0). :sweat_smile:
Having changed the log source under the logs app like you have shown me, I realized that the drop-down menu for "Field" in the Log Threshold Alert creation does not show me anything (as shown below).
I have looked at the drop-down menu for "Field" on other alert types and they work fine to me. Moreover, I have re-tried the hack with an older version of Elasticstack (7.10.1) and it worked fine there as well.
Could you have a look into this? Thank you!

First thanks for pointing out the original issue we have opened a bug here.

2nd I did my work around on 7.12.0 so it is definitely works, I did notice the drop-down is a bit hard to use / flakywith all the fields

Did you try to start typing the field name in the box? Not just the down arrow?

And you are collecting your metrics with metricbeat or the new elastic agent? And when you set the index pattern in logs viewer settings you definitely saved right?

Also curious which browser you are using?

1 Like

Hi @stephenb ,

Yes I did try to type into the field as well as trying it in different browsers (Chrome and Firefox). Moreover, yes, I'm collecting the metrics with metricbeat. Lastly, I did make sure that the log viewer settings was properly saved. Worst case scenario, I could always try to create the alert using the Alert API.

As for the workaround, I tested it on my older version of Kibana (7.10.1) and it worked fine! Thank you very much for that :+1:

Can you Remove and add a condition....

I can duplicate what you are initially seeing

WITH a chosen field

by removing all my other sources except metricbeat-* there looks to be a minor bug ...

Try adding a condition or two below that one and see if it works. That select list is very finicky in 7.12.0 .... I agree.

If I click quick it does not work if I am slow and deliberate it does

I will let them know that as well. Either way looks like the correct Match 0 : DSL Alert issue is open now.

1 Like

Hi @stephenb ,

Thank you so much for the suggestions but unfortunately they did not work on my side. However, I have successfully created the alert using the Alert APIs so for now, that should be suffice.

Again, thank you for all of the help so far! I really appreciate it. :slight_smile:

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.