Send all the aggregations as text with Watcher

alerting

(Jonhatan Ricardo Rodriguez Ramirez) #1

Hey there,
I'm configuring right now Watcher to search in the access logs and see how many error is so far and send it to a slack account.
Well, the problem that I have is because I can't know how many aggregations I will have when the query is done and in my configurations is something like "hardcoded" to send just like 5 at maximum , but if the result is grather than 5 not works.

I'm searching for 404 status code in the query and filter only for one server, then I just need send all bucket results as notification as:

Total: Total-number-of-its
Logs:
log1: number-of-results
log2: number-of-results
log3: number-of-results
log4: number-of-results
log5: number-of-results
log6: number-of-results

Here my configuration:

  "trigger" : {
    "schedule" : { "interval" : "1h" } 
  },
  "input" : {
    "search" : {
      "request": {
      "body": {
        "query": { 
          "bool": {
            "must": [
              { "range": {
                "@timestamp": {
                  "gte": "now-1h",
                  "lte": "now"
                }
                }
              }, 
              {
                "match": {
                  "beat.hostname": "someserver"
                }
              }
            ], 
            "filter": {
              "term": {
                "response": "404"
              }
            }
          }
        },
        "aggs": {
          "host": {
            "terms": {
              "field": "beat.hostname",
              "size": 1
            }
          },
          "logs_list": {
            "terms": {
              "field": "source",
              "size": 10
            }
          }
        }
       }
      }
    }
  },
    "condition": {
    "compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
  },
  "actions" : {
    "notify-slack" : {
      "throttle_period" : "30m",
      "slack" : {
        "message" : {
          "from": "Watcher",
          "to" : [ "somechannel" ],
          "attachments" : [
          {
            "title" : "400 code status found",
            "text" : "Encountered: {{ctx.payload.hits.total}} in the last hour on {{ctx.payload.aggregations.host.buckets.0.key}} \n Files: \n {{ctx.payload.aggregations.logs_list.buckets.0.key}}: {{ctx.payload.aggregations.logs_list.buckets.0.doc_count}} \n {{ctx.payload.aggregations.logs_list.buckets.1.key}}: {{ctx.payload.aggregations.logs_list.buckets.1.doc_count}} \n {{ctx.payload.aggregations.logs_list.buckets.2.key}}: {{ctx.payload.aggregations.logs_list.buckets.2.doc_count}} \n {{ctx.payload.aggregations.logs_list.buckets.3.key}}: {{ctx.payload.aggregations.logs_list.buckets.3.doc_count}} \n {{ctx.payload.aggregations.logs_list.buckets.4.key}}: {{ctx.payload.aggregations.logs_list.buckets.4.doc_count}} \n {{ctx.payload.aggregations.logs_list.buckets.5.key}}: {{ctx.payload.aggregations.logs_list.buckets.5.doc_count}}",
            "color" : "danger"
          }
        ]
        }
      }
    }
  }

Any ideas how should I pass all buckets result?
Thanks in advance, i'm using xpack, ELK and logstash.


(Alexander Reelsen) #2

Hey,

you may want to concatenate the array values of all buckets see https://www.elastic.co/guide/en/elasticsearch/reference/5.1/search-template.html#_concatenating_array_of_values

That would be sth like (on top of my head, not tested)

"text" : "Encountered: {{ctx.payload.hits.total}} in the last hour on {{ctx.payload.aggregations.host.buckets}}{{key}}{{/ctx.payload.aggregations.host.buckets}}

--Alex


(Jonhatan Ricardo Rodriguez Ramirez) #3

Thanks for your response,

Actually didn't work, i got this message:

"status": "failure",
          "reason": "GeneralScriptException[Failed to compile inline script [Encountered: {{ctx.payload.hits.total}} in the last hour on {{ctx.payload.aggregations.host.buckets}}{{key}}{{/ctx.payload.aggregations.host.buckets}}] using lang [mustache]]; nested: MustacheException[Mismatched start/end tags: null != ctx.payload.aggregations.host.buckets in query-template:1]; "
        }

I don't know what means, but I'll investigate anyways,
thanks !


With this configuration works well,
{{#ctx.payload.aggregations.logs_list.buckets}}{{key}}: {{doc_count}} {{/ctx.payload.aggregations.logs_list.buckets}}
Another guy help me with this code line, this line works to loop over your aggregation in the action.
Just in case another people need to do something like this, thanks !


(Alexander Reelsen) #4

Hey,

that message was just telling you, that the script I showed was invalid and could not be compiled (scripts are compiled for faster execution). The compilation error stemmed from the missing # sign in the first directive.

Glad you solved it and thanks for putting the right solution up here!

--Alex


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.