Watcher, X amount of logs, per X host per hour

Hello, I have been fighting with this for a few hours now and just can't get an alert that works.

I want to basically find out when a host is spamming syslog, so I have a watcher like so:

{
  "trigger": {
    "schedule": {
      "interval": "1m"
    }
  },
  "input": {
    "search": {
      "request": {
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "filter": {
                "range": {
                  "@timestamp": {
                    "gte": "{{ctx.trigger.scheduled_time}}||-1h",
                    "lte": "{{ctx.trigger.scheduled_time}}",
                    "format": "strict_date_optional_time||epoch_millis"
                  }
                }
              }
            }
          },
          "aggs": {
            "bucketAgg": {
              "terms": {
                "field": "host.name",
                "size": "1",
                "order": {
                  "_count": "desc"
                }
              }
            }
          }
        },
        "indices": [
          "system-logs-*"
        ]
      }
    }
  },
  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gte": 500
      }
    }
  }
}

However when I run this, it fails to perform any actions with the error:

"error": {
          "root_cause": [
            {
              "type": "general_script_exception",
              "reason": "Error running com.github.mustachejava.codes.DefaultMustache@8a1d0f5"
            }
          ],
          "type": "general_script_exception",
          "reason": "Error running com.github.mustachejava.codes.DefaultMustache@8a1d0f5",
          "caused_by": {
            "type": "mustache_exception",
            "reason": "Failed to get value for ctx.payload.hits.hits.0._source.host.name @[query-template:1]",
            "caused_by": {
              "type": "mustache_exception",
              "reason": "0 @[query-template:1]",
              "caused_by": {
                "type": "index_out_of_bounds_exception",
                "reason": "0"
              }
            }
          }

Am I missing something blindly obvious here or am I just not understanding how to correct persue it? I am migrating from elastalert to watcher, this is how my elastalert looked like:

# Check a single host for over 500 syslog messages in an hour
name: Spamming Syslog Messages (1 Hour) - Warning
filter:
  - term:
      source: /var/log/syslog
type: frequency
num_events: 500
timeframe:
  hours: 1
query_key: beat.hostname

# (Required)
# Index to search, wildcard supported
index: system-logs-*

terms_window_size:
  days: 1

I feel like my information is somewhat lacking so I'll just cover what my alert should do.

Check index for over 500 items that belong to:

system-logs-*

Field: [source: /var/log/syslog]

Number of documents >= 500

per host.

So if a host has over 500 entries in elasticsearch with the same source record, then it should alert.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.