Ruby Code out of Logstash Pipeline

Elastic Stack Logstash
1

Dear all

I am building a filter plugin that calls a ruby .rb code file. The filter itself is well documented.
The issues I have is with ruby gems and the transmission of variables:

The documentation says that ruby gems don't work. I however need to include "nokogiri" in the .rb file for crawling a website. When will this feature be possible? - see below.

I also need to transfer variables from logstash to .rb ("script_params" I assume). And then from .rb back to the logstash pipeline - how does this work?

Best Michael

Source: https://www.elastic.co/de/blog/moving-ruby-code-out-of-logstash-pipeline

"#### Limitations

Even though file-based scripting in the ruby filter provides code reuse and testability, there are still limitations compared to full blown Logstash plugins. The main one is lack of support for third-party libraries: if you need the custom ruby code to use a library that doesn't ship with Logstash, it's not possible to load it through the ruby filter."

2

This is not currently possible, and is not a feature that is planned to be added.

The ruby script can provide a register method that takes a key/value hash as input, and sets instance variables that can be used when the filter(event) is invoked.

I have a small collection of such scripts here that can be used as examples.

3

Dear yaauie

Thanks. I guess I need to split up the single intended logstash pipeline into multiple processes with some of them running on ruby native code.

Michael