Kibana rule for detecting lost applicattions

Elastic Stack Kibana
1

Hello,

I'm trying to create a rule in Kibana (v 8.18.7) to detect when an application has suddenly stop to inject documents in Kibana. My idea is to create a Kibana rule for that (without use transforms and Machine Learning) that checks the entities (for example applications) that injected documents one week ago but not now. I don’t know if I can use ES|QL in an Elasticsearch query rule or other kind of Kibana rule for that (comparing for example the ‘application’ field that existed in the documents injected one week ago and evaluate how many of these 'application' fields are not being injected now), even if I can count how many applications were present one week ago and how many applications are present now could be useful (if the number is not the same, I could send a notification for checking).

NOTE: When I say ‘one week ago’ could be a timeframe for example between two weeks ago and one week ago, and when I say ‘now’ could be last 24 hours.

Many thanks in advance.

Best regards.

2

What does yout documents looks like? Is the application name on the application field?

You could use a security ESQL rule to alert when the time of the last indexed document has a difference from the execution rule time higher than the desired value.

I use some rules like this to alert when some dataset stopped sending data.

The logic is like this:

FROM datastream-name
| STATS last_timestamp = MAX(@timestamp) by application
| EVAL lag = DATE_DIFF("minute", last_timestamp, NOW())
| WHERE lag >= 30
| LIMIT 100

This will alert when the last document for any value on the application field has a difference higher than 30 minutes from the time the rule was executed.

When creating the security rule, the look-back time needs to be higher than the desired lag interval.

1 Like
3

Many thanks @leandrojmp for your quick response, I will try to use something similar to the logic that you describe for my rules.