System status (Green, Yellow, Red) for specific watcher

Elastic Stack Elasticsearch
1

Hello,
I'm setting up watchers in our system and I want to know if there is a possibility to recognize status (Green, Yellow, Red) for watcher.
E.g
If connection with some service went down and we will have 3 exceptions in last 5 minutes I want to send alert with status Yellow. Then if there will be 2 more I want to change status to Red. When connection came back I want to get email with Green status.

For now I set up watcher for monitoring connection with some service and it works but I'm wondering about those statuses.

I will be really happy if someone could help me with this.

Thanks

2

Hi,

This is not an easy one because you need to keep a kind of "state" around for your service.

You could define a watch A that checks every minute the status of your remote service. If OK, no action is executed (or maybe a log debug message). If not OK, the watch index a document in a dedicated index.

You could define another watch B that retrieves every minute the documents in the dedicated index for the last 5 minutes, in order. Then using a script it can iterate over these documents and sets the status of the email accordingly?

Note that the dedicated index must be cleaned up at some point. Also, Watcher can record the watch history so maybe you can query this one to retrieve the latest watch execution status instead of using a dedicated index.

1 Like
3

I like idea of using watch history. I'll try to set this up.
Once I get this working I update this post with status :slight_smile:

Thank you!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.