Watcher Alert based on Status down for tomcat and not trigger based on time interval

Hello All,

I have a requirement where in watcher should trigger on the basis of status down and not based on
time interval settings.
Below is my working watcher script that triggers every 15 min or 8hrs accordingly set whenever any tomcat goes down.
Now as per admin feedback they take downtime for say 4 hours and shutdown tomcat then every 15 min they will get the alerts,as they are aware the down time is 4 hours so there expectation is for the very first time TOMCAT goes down they should get alert and not every 15 min of 4 hour.

I am getting tomcat data in index every 10 seconds and have field tomcat.server_Status: UP/DOWN
coming.

Simply everytime tomcat goes down I want alert, but not trigger on the basis of time interval instead of status.
Simply when tomcat goes down for very first time trigger alert and when comes up :NO alert and when again down get Alert.
The intention is not to bombard alerts every few minutes even if its known for this much amount of time tomcat will be down for DOWNTIME.

plz suggest changes ,Not sure if possible or how to do this.

{
  "trigger": {
    "schedule": {
      "interval": "15m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "mis-monitoring-webserver-*"
        ],
        "rest_total_hits_as_int": true,
        "body": {
          "aggs": {
            "host": {
              "terms": {
                "field": "host.name.keyword",
                "order": {
                  "_key": "desc"
                },
                "size": 10000
              },
              "aggs": {
                "port": {
                  "terms": {
                    "field": "tomcat.port",
                    "order": {
                      "_key": "desc"
                    },
                    "size": 10000
                  },
                  "aggs": {
                    "application_name": {
                      "terms": {
                        "field": "tomcat.application_name",
                        "order": {
                          "_key": "desc"
                        },
                        "size": 10000
                      },
                      "aggs": {
                        "status": {
                          "top_hits": {
                            "fields": [
                              {
                                "field": "tomcat.server_status"
                              }
                            ],
                            "_source": false,
                            "size": 1,
                            "sort": [
                              {
                                "@timestamp": {
                                  "order": "desc"
                                }
                              }
                            ]
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          },
          "size": 0,
          "query": {
            "bool": {
              "filter": [
                {
                  "script": {
                    "script": """if (doc['tomcat.server_status.keyword'].size() != 0 )
              {
                     def a=doc['tomcat.server_status.keyword'].value;
                     if (a=='DOWN')
                     {
                             return true;
                     }
                     else
                     {
                        return false;
                     }
}
 """
                  }
                },
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-15m",
                      "lte": "now"
                    }
                  }
                }
              ]
            }
          }
        }
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },
  "actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "to": [
          "prashan.mehta@outlook.com"
        ],
        "subject": "TOMCAT DOWN STATUS",
        "body": {
          "html": """
{{#ctx.payload.aggregations.host.buckets}}
<p><b>Summary:</b> Tomact is down on server {{key}}</p>
<p><b>Date and Time:</b> {{ctx.trigger.scheduled_time}}</p>
<p><b>Description:</b>Tomcat is down with the following details</p>
{{#port.buckets}}
- Port: {{key}}, Applications: {{#application_name.buckets}}{{key}}<br>

<p><b>Status:</b> DOWN</p>
{{/application_name.buckets}}{{/port.buckets}}
<p><b>Issued By:</b> CIS Monitoring System</p>
<hr />
{{/ctx.payload.aggregations.host.buckets}}
"""
        }
      }
    }
  }
}

THANKS

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.