Use ruby variable in if filter

[kitz99 (Bogdan Timofte)]

I have this logstash config:

input { stdin { } }

output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}

filter {
  ruby {
     code => "
         @@exists_pattern = ['/collaboration/display', '/collaboration/questions'].any?{ |pattern| event.get('message').include?(pattern) }
         event.add('keep_line', @@exists_pattern)
     "
  }

  if not [keep_line] { drop { } }


  grok {
match => {
  "message" => '%{IP:serverip} \[%{HTTPDATE:my_timestamp}\]'
}
  }

  date {
match => [ "my_timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
target => "@timestamp"
  }
}

But when I try to run logstash with this config file I get this error message:

[ERROR][logstash.agent           ] Cannot create pipeline {:reason=>"Expected one of #, ( at line 30, column 10 (byte 923) after filter {\n  # grok {\n

How can I use field 'keep_line' in that if condition?

[magnusbaeck (Magnus Bäck)]

Yes. "keep_line" isn't a Ruby variable, it's a field in the current event. I'm not sure what Logstash is complaining about. Judging by the error message there's something about the grok filter that it doesn't like. Try commenting out pieces of the configuration to narrow down the cause of the error.

[kitz99 (Bogdan Timofte)]

grok seems to be ok. If I comment it, I still get this error:

[ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, ( at line 16, column 10 (byte 350) after filter {\n ruby {\n code => "\n @@exists_pattern = ['/collaboration/display', '/collaboration/questions'].any?{ |pattern| event.get('message').include?(pattern) }\n event.add('keep_line', @@exists_pattern)\n "\n }\n\n if not "}
2017-06-29 12:05:03,847 Api Webserver ERROR No log4j2 configuration file found. Using default configuration: logging only errors to the console.

[kitz99 (Bogdan Timofte)]

The problem seems to be with not [keep_line]. I switched to if [keep_line] and logstash started. I keep digging