Logstash Ruby filter error

Elastic Stack Logstash
1

Hi All,

I have written a ruby filter to check if a pattern present in a message and drop the event if pattern is not present.

I'm getting below mentioned error after updating the logstash configuration,

[2018-07-17T23:27:55,117][ERROR][logstash.filters.ruby ] Could not process event: no implicit conversion of LogStash::Event into String {:script_path=>"/etc/logstash/conf.d/extra/scripts/avips.rb", :class=>"TypeError", :backtrace=>["org/jruby/RubyRegexp.java:1107:inmatch'", "/etc/logstash/conf.d/extra/scripts/avips.rb:5:in filter'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-ruby-3.1.4/lib/logstash/filters/ruby/script/context.rb:55:inexecute_filter'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-ruby-3.1.4/lib/logstash/filters/ruby/script.rb:30:in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-ruby-3.1.4/lib/logstash/filters/ruby.rb:98:infile_script'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-ruby-3.1.4/lib/logstash/filters/ruby.rb:84:in filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:indo_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in block in multi_filter'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:47:inmulti_filter'", "(eval):8476:in block in initialize'", "org/jruby/RubyArray.java:1734:ineach'", "(eval):8473:in block in initialize'", "(eval):1519:inblock in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:445:in filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:424:inworker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:386:in block in start_workers'"]}

Below is the code written in ruby file.
`
def register(params)
end

def filter(event)
if /^.([sid:\s\d+]).$/.match(event)
return event
else
return nil
end
end
`

Below is the logstash filter code
filter { if "av" in [tags] and "test" in [tags] { ruby { path => "/etc/logstash/conf.d/extra/scripts/avips.rb" add_tag => ["ips"] } } }

2 
if /^.([sid:\s\d+]).$/.match(event.get("message"))
    return [event]
else
    return []
end

The add_tag appears not to work, which feels like a bug to me.

3

I believe your pattern may need a little tweaking: brackets and parens ([, ], (, and )) have special meaning and need escaping to match literal characters, so your pattern becomes:

/^\(\[sid:\s\d+\]\)/

However, what you're looking to do can be achieved in normal Logstash flow control using the Drop Filter Plugin:

filter {
  if [message] !~ /^\(\[sid:\s\d+\]\)/ {
    drop { }
  }
}
4

Logstash is throwing a ruby exception. That I guess is a reason that add_tag is not working

5

I want to use regex to match a message and then add a tag accordingly. using if statement seems to be solving the problem. However, I'm still not sure, if ruby filter is returning a Logstash::Event object or array, how to convert that to a string so that logstash can process it further.

6

No, sir. Even if you fix the exception it still ignores the add_tag.

7

In general, decoration only occurs in Logstash filter plugins on success, by design. It is up to each plugin to respect this, so there may be a few that behave incorrectly, but generally that is the rule.

8

The Event is the base unit in a Logstash pipeline -- it has fields and metadata that are manipulated by the filters and used by the outputs. The Ruby filter should return an array of zero or more Events.

9

It does not decorate. Even when it succeeds.

10

:thinking: confirmed:

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.