Additional Ruby Gems to use in Logstash's ruby filter

[davidS (David)]

Hi all,

how can I install additional ruby gems in Logstash 5.3 to use it by its ruby filter plugin?
In Ruby i normally could do ./bundle with a new entry in the Gemfile or by type 'gem install GEM'

I tried also this

sudo env GEM_HOME=./vendor/bundle/jruby/1.9 ./vendor/jruby/bin/jruby ./vendor/jruby/bin/gem install public_suffix -v 1.4.6

It works but it was not find by logstash later

Bundler::GemNotFound: Could not find gem 'public_suffix (= 1.4.6) java' in any of the gem sources listed in your Gemfile or installed on this machine.

Best regards
David

[davidS (David)]

I found the issue.

I installed the gem by root but all gems must be installed by logstash's user (default: logstash)
So after

chmod -R logstash: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/

the gem was found. But maybe there is a general permission issue.

sudo -u logstash env GEM_HOME=./vendor/bundle/jruby/1.9 ./vendor/jruby/bin/jruby ./vendor/jruby/bin/jgem install public_suffix

generates

Permission denied - /usr/share/logstash/vendor/bundle/jruby/1.9/cache/public_suffix-1.4.6.gem

because of root:root as user/group settings for that directory.

Why /usr/share/logstash is generally installed for root:root?

So now if I start logstash again with public_suffix installed and permissions setuped correctly

sudo -u logstash bin/logstash -f /etc/logstash/conf.d/logstash-squid.conf --path.settings=/etc/logstash/ --log.level=debug

I got the following

ERROR FileManager (/var/log/logstash/logstash-plain.log) java.io.FileNotFoundException: /var/log/logstash/logstash-plain.log (No Permissions)

So I setup also these permissions from root:root to logstash:logstash

sudo chown -R logstash: /var/log/logstash/

and start logstash again. Next issue dropped me out of my hope to see the end of the tunnel

An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data/queue" must be a writable directory. It is not writable.

I setuped permission again also for the whole /usr/share/logstash directory.

sudo chown -R logstash: /usr/share/logstash

That's it.