Logstash-plugin install logstash-output-elasticsearch 5.1.0 installs logstash-output-elasticsearch-java 5.1.0

Elastic Stack Logstash
1

Hi,

I'm in the process of upgrading to 2.4.0 and wanted to test the new version of the output plugin to elastic.
On a fresh install installing logstash-output-elasticsearch 5.1.0 installs instead logstash-output-elasticsearch-java 5.1.0.

Has anyone else seen this?

root@vagrant-ubuntu-trusty-64:/# dpkg-query --showformat='${Version}\n' --show logstash
1:2.4.0-1
root@vagrant-ubuntu-trusty-64:/# java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
root@vagrant-ubuntu-trusty-64:/#

root@vagrant-ubuntu-trusty-64:/# DEBUG=1 /opt/logstash/bin/logstash-plugin install --version 5.1.0 'logstash-output-elasticsearch'
DEBUG: exec /opt/logstash/vendor/jruby/bin/jruby --1.9 -J-XX:+UseParNewGC -J-XX:+UseConcMarkSweepGC -J-Djava.awt.headless=true -J-XX:CMSInitiatingOccupancyFraction=75 -J-XX:+UseCMSInitiatingOccupancyOnly -J-XX:+HeapDumpOnOutOfMemoryError -J-Xmx1g -J-XX:HeapDumpPath=/opt/logstash/heapdump.hprof /opt/logstash/lib/pluginmanager/main.rb install --version 5.1.0 logstash-output-elasticsearch
Validating logstash-output-elasticsearch-5.1.0
Installing logstash-output-elasticsearch
[....]
Installing logstash-output-elasticsearch 5.1.0 (was 2.7.1)
0: logstash-output-elasticsearch (5.1.0) from /opt/logstash/vendor/bundle/jruby/1.9/specifications/logstash-output-elasticsearch-5.1.0-java.gemspec

Looking for the installed plugin:

root@vagrant-ubuntu-trusty-64:/opt/logstash# /opt/logstash/bin/logstash-plugin list --verbose --installed | grep elastic
logstash-input-elasticsearch (2.0.5)
logstash-output-elasticsearch (5.1.0)

but

root@vagrant-ubuntu-trusty-64:/opt/logstash# find . -name 'logstash-output-elasticsearch-5.1.0*'
./vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-5.1.0-java
./vendor/bundle/jruby/1.9/specifications/logstash-output-elasticsearch-5.1.0-java.gemspec
./vendor/bundle/jruby/1.9/cache/logstash-output-elasticsearch-5.1.0-java.gem
root@vagrant-ubuntu-trusty-64:/opt/logstash#

Thanks!
Gabriele

2

logstash-output-elasticsearch-java ≠ logstash-output-elasticsearch_java

The hyphen - differs from the underscore _ version in that -java means it's a JRuby gem, and _java means that it is using the java transport protocol (which is no longer recommended).

In other words, you can configure it and it should work as you would expect the HTTP protocol plugin version to work.

3

Thank you @theuntergeek for the quick reply :slight_smile:

By looking here All versions of logstash-output-elasticsearch | RubyGems.org | your community gem host I see they started appending -java to the version after 3.0, which was the version I was running.

I'm running now into issues with the offline installation (after pack / unpack), where plugin-install tries to reach out on the Internet on a machine which doesn't have Internet connection.
Is this caused by an updated version of bundler? Are you able to install plugins offline with 2.4?

root@logstash001:/opt/logstash/plugins# date ; DEBUG=1 /opt/logstash/bin/logstash-plugin install --local --version 5.1.0 logstash-output-elasticsearch
Wed Sep 7 08:12:27 CEST 2016
DEBUG: exec /opt/logstash/vendor/jruby/bin/jruby --1.9 -J-XX:+UseParNewGC -J-XX:+UseConcMarkSweepGC -J-Djava.awt.headless=true -J-XX:CMSInitiatingOccupancyFraction=75 -J-XX:+UseCMSInitiatingOccupancyOnly -J-XX:+HeapDumpOnOutOfMemoryError -J-Xmx1g -J-XX:HeapDumpPath=/opt/logstash/heapdump.hprof /opt/logstash/lib/pluginmanager/main.rb install --local --version 5.1.0 logstash-output-elasticsearch
Installing logstash-output-elasticsearch
[...]
Installing logstash-output-elasticsearch 5.1.0 (was 2.7.1)
0: logstash-output-elasticsearch (5.1.0) from /opt/logstash/vendor/bundle/jruby/1.9/specifications/logstash-output-elasticsearch-5.1.0-java.gemspec
[...]
HTTP GET https://bundler.rubygems.org/api/v1/dependencies
Net::HTTP::Persistent::Error: connection refused: bundler.rubygems.org:443
/opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/vendor/net/http/persistent.rb:641:in connection_for' /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/vendor/net/http/persistent.rb:994:in request'
/opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/fetcher.rb:302:in request' /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/fetcher.rb:272:in fetch'
/opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/fetcher.rb:244:in use_api' [...] /opt/logstash/lib/pluginmanager/main.rb:43:in (root)'
Fetching source index from https://rubygems.org/
Gem::RemoteFetcher::FetchError: Errno::ECONNREFUSED: Connection refused - Connection refused (https://rubygems.org/specs.4.8.gz)
/opt/logstash/vendor/jruby/lib/ruby/shared/rubygems/remote_fetcher.rb:292:in fetch_path' /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/rubygems_integration.rb:516:in fetch_specs'
/opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/rubygems_integration.rb:530:in fetch_all_remote_specs' /opt/logstash/vendor/jruby/lib/ruby/shared/rubygems/source_list.rb:90:in each'
org/jruby/RubyArray.java:1613:in each' [...] Retrying source fetch due to error (2/3): Bundler::HTTPError Could not fetch specs from https://rubygems.org/ Gem::RemoteFetcher::FetchError: Errno::ECONNREFUSED: Connection refused - Connection refused (https://rubygems.org/specs.4.8.gz) /opt/logstash/vendor/jruby/lib/ruby/shared/rubygems/remote_fetcher.rb:292:in fetch_path'
[...]
Retrying source fetch due to error (2/3): Bundler::HTTPError Could not fetch specs from https://rubygems.org/
Gem::RemoteFetcher::FetchError: Errno::ECONNREFUSED: Connection refused - Connection refused (https://rubygems.org/specs.4.8.gz)
/opt/logstash/vendor/jruby/lib/ruby/shared/rubygems/remote_fetcher.rb:292:in fetch_path' [...] /opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'
/opt/logstash/lib/pluginmanager/main.rb:43:in `(root)'
Bundler::HTTPError: Could not fetch specs from https://rubygems.org/
fetch_all_remote_specs at /opt/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/fetcher.rb:365

And after 10 minutes the new version of the plugin isn't installed:

root@logstash001:/opt/logstash/plugins# /opt/logstash/bin/logstash-plugin list --verbose | grep elastic
logstash-input-elasticsearch (2.0.5)
logstash-output-elasticsearch (2.7.1)
root@logstash001:/opt/logstash/plugins#

Thanks again!
Gabriele

4

Perhaps you could add some comments to this issue. We're having a hard time reproducing the error you describe.

Also, as it's a different topic from the original post, please ask it in a new topic.