Metrics filter stopped working after upgrade to logstash 2.2.x

Elastic Stack Logstash
#1

hi,

have anyone faced issue that logstash metrics stopped working after upgrade to 2.2.x?

it does not work both on 2.2.0 and 2.2.2

config is so (see below), also there is another ouptut section (to redis) in another file:

filter {

if [type] == "web_requests" {
    indent preformatted text by 4 spaces`if [response]  =~ /^5\d\d/ or [response]  =~ /^4\d\d/   {
        metrics {
            add_tag => "logstash_alarm"
            meter => "web_errors"
         }
    }
}

}

output {

   if "logstash_alarm" in [tags] and [web_errors.rate_1m] > 1 {
      pagerduty {
        description => "NUMBER OF 5xx ERRORS FATAL"
        details => {
          "timestamp" => "%{@timestamp}"
        }
        service_key => "xxxxxxx"
        incident_key => "logstash/servicename"
      }
    }

}

P.S.

pagerduty ouput works by itself

#2

Define stopped working, what is/is not happening, what are you expecting and seeing?

#3

it means that when condition of metrics filter is met, nothing is happening, I mean of this filter:

filter {

     if [type] == "web_requests" {
        if [response] =~ /^5\d\d/ or [response] =~ /^4\d\d/ {
            metrics {
                add_tag => "logstash_alarm"
                 meter => "web_errors"
            }
         }
    }

 }

also, even if I simplify the rule to this:

 filter {


    metrics {
        add_tag => "logstash_alarm"
        meter => "web_errors"
     }
 }

 output {

     if "logstash_alarm" in [tags] and [web_errors.rate_1m] > 1 {
         pagerduty {
             description => "NUMBER OF 5xx ERRORS FATAL"
             details => {
                 "timestamp" => "%{@timestamp}"
             }
             service_key => "xxxxxxx"
             incident_key => "logstash/servicename"
         }
       }

 }

also nothing is happening (when condition is met)

but if I throw out the filter metrics:

 output {
     pagerduty {
         description => "NUMBER OF 5xx ERRORS FATAL"
         details => {
              "timestamp" => "%{@timestamp}"
          }
         service_key => "xxxxxxx"
         incident_key => "logstash/servicename"
         }

 }

it work by itself, so the problem is in metrics filter, not in pagerduty

#4

You are mistaking in [web_errors.rate_1m] part. Documentation for plugin (https://www.elastic.co/guide/en/logstash/current/plugins-filters-metrics.html) says that in Logstash 2.x [web_errors][rate_1m] must be used instead.

I also suggest to change if [response] =~ /^5\d\d/ or [response] =~ /^4\d\d/ to numerical comparison if possible. Adding :int qualifier to your grok pattern for response should be enough. Check more about type conversions in https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html.

1 Like
#5