Can't make conditionnal filtering to work

Elastic Stack Logstash
1

Hello,

I plan on migrating to logstash (from graylog) and I'm facing a problem with the conditionnal filtering. Without the condition, the grok pattern works fine and as expected. However, if I put the condition, it will stop parsing completely.
Here's the code :

input {
  beats {
    port => 5048
    type => "test"
  }
}
filter {
  if [type] == "test" {
    grok {
      match => { "message" => "%{NUMBER:duration}\s%{IP:client_address}\s%{WORD:cache_result}/%{POSINT:status_code}\s%{NUMBER:bytes}\s%{WORD:request_method}\s%{NOTSPACE:url}\s(%{NOTSPACE:user}|-)\s%{WORD:hierarchy_code}/%{IPORHOST:server}\s%{NOTSPACE:content_type}" }
    }
  }
}

Any insights on what I'm doing wrong ?

Thanks.

2

What does an event look like? Use a stdout { codec => rubydebug } output to inspect it.

4

It seems that I misunderstood how logstash works, and can't put "type => "thing" " in the input.
I tried the condition with some existing field like host or type and it worked fine.

But it seems that it doesn't work for any field. I tried to use a condition on beat.name (generated by filebeat if i'm not mistaken) andit didn't work. Also tried tags, same result.

Host will do for now but it would be nice to understand why it doesn't work for some fields.

5

If you show us what you tried we can probably tell you why it didn't work.

6 
input {
  beats {
    port => 5048
  }
}
filter {
  if [host] == "servername" {
    grok {
      match => { "message" => "%{NUMBER:duration}\s%{IP:client_address}\s%{WORD:cache_result}/%{POSINT:status_code}\s%{NUMBER:bytes}\s%{WORD:request_method}\s%{NOTSPACE:url}\s(%{NOTSPACE:user}|-)\s%{WORD:hierarchy_code}/%{IPORHOST:server}\s%{NOTSPACE:content_type}" }
    }
  }
}

Works

input {
  beats {
    port => 5048
  }
}
filter {
  if [tags] == "squid3" {
    grok {
      match => { "message" => "%{NUMBER:duration}\s%{IP:client_address}\s%{WORD:cache_result}/%{POSINT:status_code}\s%{NUMBER:bytes}\s%{WORD:request_method}\s%{NOTSPACE:url}\s(%{NOTSPACE:user}|-)\s%{WORD:hierarchy_code}/%{IPORHOST:server}\s%{NOTSPACE:content_type}" }
    }
  }
}

Doesn't work. The tags was specified in the filebeat conf file. Doesn't work either with name(shipper name in the filebeat conf file), beat.name, fields created by the "field" option in filebeat. Doesn't work with port either :

input {
  beats {
    port => 5048
  }
}
filter {
  if [port] == 5048 {
    grok {
      match => { "message" => "%{NUMBER:duration}\s%{IP:client_address}\s%{WORD:cache_result}/%{POSINT:status_code}\s%{NUMBER:bytes}\s%{WORD:request_method}\s%{NOTSPACE:url}\s(%{NOTSPACE:user}|-)\s%{WORD:hierarchy_code}/%{IPORHOST:server}\s%{NOTSPACE:content_type}" }
    }
  }
}
7 
if [tags] == "squid3" {

tags is an array that might contain multiple tags. Use this instead:

 if "squid3" in [tags] {

Doesn't work with port either

Does the port field actually exist? And if so, is it an integer and not a string?

8

You're right, tags was an array, your solution resolved the problem.

Port isn't indeed, I made a mistake.

I will go with the tags condition I guess. Thanks.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.