I can not install any LogStash plug-in in ES 5.0

Elastic Stack Logstash
1

I did a fresh install of ES 5.0.

When trying to install LS plug-ins like logstash-filter-multiline and logstash-filter-mutate I keep getting the following error message:
An error occurred while installing logstash-core-event-java <5.0.0> , and Bundler can not continue.
Make sure that 'gem instal logstash-core-event-java -v '5.0.0'' succeeds before bundling.

Any hints as to what may be causing this error would be much appreciated.

2 Likes
2

I have the same issue. I went through the process of installing jruby (not a requirement as far as I can see) and it still didn't work.

I tried building the gem directly, but I get a lot of different errors that way. For all I know it requires a specific version of ruby and/or jruby.

3

Following the doc exactly shows that you're supposed to run this from the install directory which for RHEL is /usr/share/logstash. So there is a bundled jruby binary, but the gem build crashes.

Fetching: ruby-maven-3.3.9.gem (100%)
Successfully installed ruby-maven-3.3.9
jar dependencies for logstash-core-event-java-5.0.0-java.gemspec . . .
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-parseable POM /usr/share/logstash/vendor/jruby/lib/ruby/shared/jars/gemspec_pom.rb: only whitespace content allowed before start tag and not # (position: START_DOCUMENT seen #... @1:1) @ line 1, column 1
@
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project (/usr/share/logstash/vendor/jruby/lib/ruby/shared/jars/gemspec_pom.rb) has 1 error
[ERROR] Non-parseable POM /usr/share/logstash/vendor/jruby/lib/ruby/shared/jars/gemspec_pom.rb: only whitespace content allowed before start tag and not # (position: START_DOCUMENT seen #... @1:1) @ line 1, column 1 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/ModelParseException
ERROR: While executing gem ... (Errno::ENOENT)
No such file or directory - /usr/share/logstash/vendor/jruby/lib/ruby/gems/shared/gems/logstash-core-event-java-5.0.0-java/deps.lst

At first glance, it seems that the gem is missing a file. Not sure though if that's the extent of the problem.

4

Can you show your complete command where you are running logstash-plugin?

5

Sure, I was running this from within the /usr/share/logstash directory:

./bin/logstash-plugin install logstash-input-beats

However this appears to be a known issue, possibly with jruby or some component of it. It seems that some part of the build isn't honoring proxy settings, although even downloading and trying to build the plugin locally didn't work either.

6

And I've just confirmed this. From a server with full internet access the plugin built without any errors.

There was a fix for Ubuntu where you just had to run a command to update certs, but that command doesn't exist on Fedora-based linux.

7

I'm confused. Why do you think this is an SSL certificate problem? It doesn't look like one, to me.

8

Let's step back for a moment. I have some questions --

  1. What command(s) did you run to install logstash?
  2. What operating system and version are you on?
9

I'm on Red Hat Enterprise version 7. Logstash is installed via RPM. And I'm not sure why this was solved previously for Ubuntu with the update-ca-certs command. If you follow the Issue link, the core problem is that something involved with the plugin installation isn't honoring OS proxy settings. It didn't seem like an Elastic problem, more like something with jruby's gem build process.

Our logstash servers are not open to the internet, so I have to rely on proxy to get access to anything and I have to specify what sites I need access to.

10

Downloaded and extracted the logstash 5 TAR.

command used to install the plugin: bin\logstash-plugin
install logstash-filter-translate

os: windows 7 Professional SP1

If I try to install the gem as suggested in the error message, then I get the following error:

jar dependencies for logstash-core-event-java-5.0.0-java.gemspec . . .
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-parseable POM H:\Development\ELK5\logstash-5.0.0\vendor\jruby\lib\ruby\shared\jars\gemspec_pom.rb: only whitespace content allowed before start tag and not # (position: START_DOCUMENT seen #... @1:1) @ line 1, column 1
@
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR] The project (H:\Development\ELK5\logstash-5.0.0\vendor\jruby\lib\ruby\shared\jars\gemspec_pom.rb) has 1 error
[ERROR] Non-parseable POM H:\Development\ELK5\logstash-5.0.0\vendor\jruby\lib\ruby\shared\jars\gemspec_pom.rb: only whitespace content allowed before start tag and not # (position: START_DOCUMENT seen #... @1:1) @ line 1, column 1 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/ModelParseException

11

Is there already a solution to this?
I'm experiencing the same issue. I'm behind a proxy.

Thanks.

12

I'm also having this problem, but I am on an isolated network with no Internet access. I have to burn anything I want to CD/DVD and transfer it to my network. Any ideas would be greatly appreciated.

Terry

13

I found the solution when using proxy.
Two articles:


Need to configure proxy settings in:
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/ruby-maven-libs-3.3.9/maven-home/conf/settings.xml

I Also tried advise from

but not sure it had an impact.
(add export JAVA_OPTS="$JAVA_OPTS -Dhttp.proxyHost=HOST -Dhttp.proxyPort=PORT"
in /usr/share/logstash/bin/logstash-plugin before the ruby_exec)

14

Same problem here. I tried everything suggested above.

On logstash 5.0.1 installed behind a proxy on a windows server I tried : logstash-plugin.bat install logstash-filter-elapsed
and it failed:
> C:\logstash-5.0.1\bin>logstash-plugin.bat install logstash-filter-elapsed

Validating logstash-filter-elapsed
Installing logstash-filter-elapsed
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions
for gem "logstash-core-plugin-api":
In snapshot (Gemfile.lock):
logstash-core-plugin-api (= 2.1.16)
In Gemfile:
logstash-devutils (~> 1.1) java depends on
logstash-core-plugin-api (~> 2.0) java
logstash-input-s3 (>= 0) java depends on
logstash-mixin-aws (>= 0) java depends on
logstash-core-plugin-api (<= 2.99, >= 1.60) java
[...]
logstash-filter-elapsed (>= 0) java depends on
logstash-core-plugin-api (~> 1.0) java
logstash-core-plugin-api (>= 0) java
Running bundle update will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "logstash-core":
In snapshot (Gemfile.lock):
logstash-core (= 5.0.1)
In Gemfile:
logstash-core-plugin-api (>= 0) java depends on
logstash-core (= 5.0.1) java
logstash-filter-elapsed (>= 0) java depends on
logstash-core (< 2.0.0, >= 1.4.0) java

logstash-core (>= 0) java

Running bundle update will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "logstash":
In Gemfile:
logstash-filter-elapsed (>= 0) java depends on
logstash (< 2.0.0, >= 1.4.0) java
Could not find gem 'logstash (< 2.0.0, >= 1.4.0) java', which is required by gem
'logstash-filter-elapsed (>= 0) java', in any of the sources.

15

I just tried home without any proxy and I'm getting the same error under windows 10 with a fresh logstash download.

16

My VM(RHEL-6) is also behind the proxy and have gone through the similar kind of issues when I am trying to install logstash-filter-yaml plugin. Seems both online and offline methods are failing.

17

Same issue on RHEL 7 and Centos7 in Logstash 5.1 (logstash-5.1.1-1.noarch)

-bash-4.2$ bin/logstash-plugin install logstash-filter-translate
Validating logstash-filter-translate
Installing logstash-filter-translate
Error Bundler::InstallError, retrying 1/10
An error occurred while installing logstash-core-event-java (5.1.1), and Bundler cannot continue.
Make sure that gem install logstash-core-event-java -v '5.1.1' succeeds before bundling.
WARNING: SSLSocket#session= is not supported

18

Same issue on RHEL 6 with fresh install of logstash-5.1.1

command to install plugin:
export HTTP_PROXY=http://proxy.local
bin/logstash-plugin install logstash-filter-translate
Validating logstash-filter-translate
Installing logstash-filter-translate
which: no javac in (/usr/local/sbin:/usr/sbin:/sbin:/home/logstash/bin:/usr/bin/java:/home/logstash/bin:/usr/bin/java)
Error Bundler::InstallError, retrying 1/10
An error occurred while installing logstash-core-event-java (5.1.1), and Bundler cannot continue.
Make sure that gem install logstash-core-event-java -v '5.1.1' succeeds before bundling.

19

Hey,

same here on a Ubuntu 16.04.1 LTS box for a fresh logstash-5.1.1 installation:

/usr/share/logstash/bin/logstash-plugin install logstash-filter-elasticsearch
Validating logstash-filter-elasticsearch
Installing logstash-filter-elasticsearch
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000f4100000, 74448896, 0) failed; error='Cannot allocate memory' (errno=12)
Error Bundler::InstallError, retrying 1/10
An error occurred while installing logstash-core-event-java (5.1.1), and Bundler cannot continue.
Make sure that `gem install logstash-core-event-java -v '5.1.1'` succeeds before bundling.
WARNING: SSLSocket#session= is not supported
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000f4300000, 78118912, 0) failed; error='Cannot allocate memory' (errno=12)
20

I see you get an sslSocket warning. I also saw it in one of my (many) attempts.
It could be ssl related: ie that the proxy doesn't allow the use of https?