Unable to add logstash-input-google_cloud_storage plugin

Elastic Stack Logstash
1

I want to use logstash-input-google_cloud_storage plugin in one of our requirrement. I am using logstash-7.6 in windows 10 machine.

I followed the steps below to install the gem file, but no luck. Getting error during execution of gem install command.

Used command > gem install logstash-input-google_cloud_storage

Find error like Errno::ESRCH: No such process - No message available
chmod at org/jruby/RubyFile.java:646

Finally download the gem file and put it into local path and execute the command, and it throws error like above one.

bin/logstash-plugin install C:/Users/ThinkPad/local-gem/logstash-input-google_cloud_storage-0.10.0-java.gem

ELK\logstash-7.6.0\bin>logstash-plugin install logstash-input-google_cloud_storage
Validating logstash-input-google_cloud_storage
Installing logstash-input-google_cloud_storage
Error Bundler::InstallError, retrying 1/10
Errno::ESRCH: No such process - No message available
An error occurred while installing logstash-input-google_cloud_storage (0.11.1), and Bundler cannot continue.
Make sure that gem install logstash-input-google_cloud_storage -v '0.11.1' --source 'https://rubygems.org/' succeeds before bundling.

After that , I was installed ruby 2.6 and try to execute gem command, still the same issue persists.Ruby version is
ruby 2.6.6p146 (2020-03-31 revision 67876) [x64-mingw32]

Finally Checked the gem list and updated it using the command

gem update --system

But I couldn't find any logstash gem there.

Please suggest, whether i am following correct procedure to add google storage plugin or not.

Thank you.

2

Hello,

I do not think you need a ruby environment to install the plugin.

Open a Windows CMD prompt and ensure you JAVA_HOME is defined.

Then go to the Logstash path and run:

Z:\Downloads\logstash-7.6.0\bin>logstash-plugin install logstash-input-google_cloud_storage
Validating logstash-input-google_cloud_storage
Installing logstash-input-google_cloud_storage
Installation successful

The procedure is detailed here.

I think your ruby environment is somehow conflictin with Logstash?

3

Hi,

Thank you for your response. The error is showing like below.

[2020-04-24T18:06:36,347][ERROR][logstash.plugins.registry] Problems loading a plugin with {:type=>"input", :name=>"google_cloud_storage", :path=>"logstash/inputs/google_cloud_storage", :error_message=>"\n\n\tyou might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars with lock_jars command\n\nno such file to load -- com/google/apis/google-api-services-storage/v1-rev20181109-1.27.0/google-api-services-storage-v1-rev20181109-1.27.0 (LoadError)", :error_class=>RuntimeError, :error_backtrace=>["uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:356:in do_require'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:265:in block in require_jar'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:307:in require_jar_with_block'", "uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/jar_dependencies.rb:264:in require_jar'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/lib/bootstrap/patches/jar_dependencies.rb:6:in require_jar'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash-input-google_cloud_storage_jars.rb:12:in '", "org/jruby/RubyKernel.java:978:in require'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/cloud_storage/client.rb:5:in <main>'", "org/jruby/RubyKernel.java:978:in require'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/google_cloud_storage.rb:6:in '", "org/jruby/RubyKernel.java:978:in require'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/vendor/bundle/jruby/2.5.0/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/logstash-core/lib/logstash/plugins/registry.rb:191:in legacy_lookup'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/logstash-core/lib/logstash/plugins/registry.rb:166:in block in lookup'", "org/jruby/ext/thread/Mutex.java:164:in synchronize'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/logstash-core/lib/logstash/plugins/registry.rb:162:in lookup'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/logstash-core/lib/logstash/plugins/registry.rb:216:in lookup_pipeline_plugin'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/logstash-core/lib/logstash/plugin.rb:144:in lookup'", "org/logstash/plugins/PluginFactoryExt.java:181:in plugin'", "org/logstash/execution/JavaBasePipelineExt.java:60:in initialize'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/logstash-core/lib/logstash/java_pipeline.rb:27:in initialize'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/logstash-core/lib/logstash/pipeline_action/create.rb:36:in execute'", "C:/Users/ThinkPad/Saroj/ELK/logstash-7.6.0/logstash-core/lib/logstash/agent.rb:326:in `block in converge_state'"]}

Please advice.

4

Hi,

Logstash-7.6 will support jdk 1.8 or not. I have installed jdk 11.0.7. still same issue to download the plugin.Its showing some ruby gem error.

5

Hi,

Please find the step, i have followed here

C:\Users\ThinkPad>echo %JAVA_HOME%
C:\Program Files\Java\jdk-11.0.7

C:\Users\ThinkPad>cd C:\Users\ThinkPad\Saroj\ELK\ELK_7.6\logstash-7.6.0\bin

C:\Users\ThinkPad\Saroj\ELK\ELK_7.6\logstash-7.6.0\bin>logstash-plugin install logstash-input-google_cloud_storage
2020-04-25T12:50:19.805+01:00 [main] WARN FilenoUtil : Native subprocess control requires open access to sun.nio.ch
Pass '--add-opens java.base/sun.nio.ch=org.jruby.dist' or '=org.jruby.core' to enable.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.jruby.ext.openssl.SecurityHelper (file:/C:/Users/ThinkPad/Saroj/ELK/ELK_7.6/logstash-7.6.0/vendor/jruby/lib/ruby/stdlib/jopenssl.jar) to constructor java.security.cert.CertificateFactory(java.security.cert.CertificateFactorySpi,java.security.Provider,java.lang.String)
WARNING: Please consider reporting this to the maintainers of org.jruby.ext.openssl.SecurityHelper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Validating logstash-input-google_cloud_storage
Installing logstash-input-google_cloud_storage
Error Bundler::InstallError, retrying 1/10
Errno::ESRCH: No such process - No message available
An error occurred while installing logstash-input-google_cloud_storage (0.11.1), and Bundler cannot continue.
Make sure that gem install logstash-input-google_cloud_storage -v '0.11.1' --source 'https://rubygems.org/' succeeds before bundling.

In Gemfile:
logstash-input-google_cloud_storage

Please suggest

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.