Drop filter is not dropping the logs

Elastic Stack Logstash
1

I need to drop logs when syslog5424_host is e.g dev.dev.sample-app-109, the number at the end will change but I have regex to accommodate that. This is my current configuration but it's not dropping the logs.

The field syslog5424_host looks something like this:
dev.dev.sample-app-206, dev.dev.sample-app-206, dev.dev.sample-app-206

filter {
  if [syslog5424_host] =~ /(dev\.dev\.sample-app-[0-9]{1,10},?\s?)+/ {
      mutate {
      add_tag => ["SampleApp"]
    }
  }

  if "SampleApp" in [tags] {
    drop { }
  }
}
2

If you use

output { stdout { codec => rubydebug } }

then what do the [syslog5424_host] and [tags] fields look like? (I am wondering if [syslog5424_host] could be an array.)

3

Sorry for replying so late I broke my ELK setup, yeah you are right its a array, how do I deal with array?

"syslog5424_host" => [
Aug 05 10:19:52 dc-elk-pr-log02 logstash[10169]:         [0] "dev.dev.sample-app-206",
Aug 05 10:19:52 dc-elk-pr-log02 logstash[10169]:         [1] "dev.dev.sample-app-206",
Aug 05 10:19:52 dc-elk-pr-log02 logstash[10169]:         [2] "dev.dev.sample-app-206"
Aug 05 10:19:52 dc-elk-pr-log02 logstash[10169]:     ],
4

If there are always exactly three host entries then you could test each of them using

if [syslog5424_host][0] =~ /dev\.dev\.sample-app-[0-9]{1,10},?\s?/

etc. Alternatively, do it in ruby. Something like this (which I have not tested)

ruby {
    code => '
        matched = true
        hosts = event.get("syslog5424_host")
        if hosts.is_a? Array && hosts.length > 0
            hosts.each { |x|
                matched &= x.match?(/dev\.dev\.sample-app-[0-9]{1,10},?\s?/)
            }
            if matched { event.cancel }
        end
    '
}
5

Awesome!!! thank you so much

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.