Conditional output to statsd

Elastic Stack Logstash
1

Hi,

I'm trying to use logstash (v 1.4.5) to increment a counter in statsd, based on a pattern match in an haproxy httplog file.

I am already sending various metrics from each line in the file, but I'm now trying to add a count of the number of lines where the path requested matches /important/path. I'm currently trying to do this by adding a tag and then a conditional output which only comes into effect if that tag is detected.

At the moment I'm struggling with getting it to work, so I though I would ask for help. I have checked in the rubydebug output that the pattern matching and tagging appears to work, but I'm getting no output to statsd, as far as I can tell so far.

If anyone has any insight on how I might get this to work, I'd be grateful.

My configuration looks like this:

input {
  file {
    type => "haproxy"
      path => "/var/log/haproxy/haproxy.log"
  }
}

filter {
  if [type] == "haproxy" {
    grok { 
      match => { "message" => "%{HAPROXYHTTP}" }
    }
    if [http_request] =~ /^\/important\/path$/ {
      mutate { add_tag => "tagged_important" }
    }
  }
}

output {
  if  [type] == "haproxy" {
    statsd {
      host => "statsd-host"
      count => [
        "haproxy.%{frontend_name}.%{backend_name}.%{server_name}.response_size", "%{bytes_read}"
      ]
      increment => [
        "haproxy.%{frontend_name}.%{backend_name}.%{server_name}.hits",
        "haproxy.%{frontend_name}.%{backend_name}.%{server_name}.responses.%{http_status_code}"
      ]
      timing => [
        "haproxy.%{frontend_name}.%{backend_name}.%{server_name}.response_time", "%{time_duration}",
        "haproxy.%{frontend_name}.%{backend_name}.%{server_name}.queue_time", "%{time_queue}",
         <snip snip more like this>
        "haproxy.%{frontend_name}.%{backend_name}.%{server_name}.response_size", "%{bytes_read}"
      ]
    }
    if [tagged_important] in [tags] {
      statsd {
        host => "statsd-host"
        increment => [
          "haproxy.important_request"
        ]
      }
    }
    stdout { codec => rubydebug }
  }
}
2

Oh, I think I have fixed it.

My conditional output test should have been a quoted string, rather than encased in square brackets.
i.e.

if "tagged_important" in [tags]

...not

if [tagged_important] in [tags]

Any other suggestions on how people achieve this kind of thing would be welcome.

3

Looks alright to me. Could you try replacing the tagged_important statsd output with e.g. a file output (or move the existing stdout output a few lines below to being inside that conditional) to remove a possible error source?

1 Like
4

Thanks Magnus.
It's working now.
I've got a load more conditional outputs that I can implement, now that I know how to do it properly.