How to track failure or exception?

[nobound]

Hi,

In the logstash configuration file, I use grok, ruby and aggregate multiple times within the filter section. When I check for _grokparsefailure, _aggregateexception, _rubyexception (whatever do I need to check?) at the end, how I can tell where the failure comes from?
I could check error after each filter plugin, but would like to know if there is a better way to do so.

Thanks

[yaauie (Ry Biesemeyer)]

There are two ways:

1. configure the plugin to tag the event with something specific and descriptive when it fails
2. handle failures close to the bits that can cause failure

The below example does both:

filter {
  grok {
    id => "descriptive unique id"
    tag_on_timeout => "your timeout tag for this instance"
    tag_on_failure => ["your failure tag for this instance"]
    match => {
      # ...
    }
  }
  if [tags] include "your timeout tag for this instance" {
    # ... handle timeouts. these events consume a _lot_ of resources
  } else if [tags] include "your failure tag for this instance" {
    # handle failures to parse. 
  }
}