Logstash-filter-zeromq does not install (version conflict) on logstash 5.1

Elastic Stack Logstash
1

Hi all,

I have trouble installing the logstah-filter-zeromq plugin on a clean logstash 5.1 install (tested with docker hub docker image logstash:5.1). Installation reports a version conflict. Full console output following below. The command used for installation is

logstash-plugin install logstash-filter-zeromq

Any advice is highly appreciated. Thanks in advance!

Console output [truncated due to message size limit]:

   Validating logstash-filter-zeromq
Installing logstash-filter-zeromq
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core":
  In snapshot (Gemfile.lock):
    logstash-core (= 5.1.1)

  In Gemfile:
    logstash-core-plugin-api (>= 0) java depends on
      logstash-core (= 5.1.1) java

    logstash-filter-zeromq (>= 0) java depends on
      logstash-core (< 2.0.0, >= 1.4.0) java

    logstash-core (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "logstash-core-plugin-api":
  In snapshot (Gemfile.lock):
    logstash-core-plugin-api (= 2.1.12)

  In Gemfile:
    logstash-devutils (>= 0) java depends on
      logstash-core-plugin-api (~> 2.0) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

   [...]

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-output-file (>= 0) java depends on
      logstash-core-plugin-api (< 2.99, >= 2.0.0) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-output-stdout (>= 0) java depends on
      logstash-core-plugin-api (< 2.99, >= 1.60.1) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-filter-zeromq (>= 0) java depends on
      logstash-core-plugin-api (~> 1.0) java

    logstash-core-plugin-api (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "logstash":
  In Gemfile:
    logstash-filter-zeromq (>= 0) java depends on
      logstash (< 2.0.0, >= 1.4.0) java
Could not find gem 'logstash (< 2.0.0, >= 1.4.0) java', which is required by gem 'logstash-filter-zeromq (>= 0) java', in any of the sources.
ERROR: Service 'logstash' failed to build: The command '/bin/sh -c logstash-plugin install logstash-filter-zeromq' returned a non-zero code: 1
2

The issue is not docker specific but also happens on a clean Ubuntu 16.04 (using opejdk-8 jre). Any advice is highly appreciated!
Alternatively: Is there any other method recommended to send logstash events to an external service for filtering?

3

Now also tried to build the plugin locally and get a more concise error message. It seems that the core-plugin-api and the zeromq mixin seem to require different versions of logstash-core. Is there any way to work aorund this?

ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core":
In snapshot (Gemfile.lock):
  logstash-core (= 5.1.1)

In Gemfile:
  logstash-core-plugin-api (>= 0) java depends on
    logstash-core (= 5.1.1) java

  logstash-filter-zeromq (= 3.0.0) java depends on
    logstash-mixin-zeromq (>= 2.0.1) java depends on
      logstash-core (< 3.0.0, >= 2.0.0) java

  logstash-core (>= 0) java
4

It looks like the zeromq plugin isn't compatible with Logstash 5.

5

Thanks for the answer! Is this really an incompatibility in terms of function?
Because from what I can judge the necessary logstash-mixin-zeromq is uploaded to rubygems.org only in version 2.0.2.
When I download the logstash-mixin-zeromq locally it is built succesfully and generates a version 3.0.0 with a dependency on logstash core that is suitbale for 5.11, and therefore can be installed.
I am currently struggling to get everything installed in my local logstash so can not (yet) make any assumptions if it will work properly

6

It could very well be that the maintainer of that plugin only needs to make a new release.

7

For further reference: The issue seems to known and is tracked on github at https://github.com/logstash-plugins/logstash-mixin-zeromq/issues/8

8

Final comment: I can confirm now that the problem is only caused by an outdated logstash-mixin-zeromq which is not "build against" the current logstash version.

9

@magnusbaeck Any chance you can prod the maintainer of zeromq plugins to take a look at this? -- Bud

10

Daniel, Were you able to build this on your own to get it to work? If so, can you provide a recipe? -- Bud

11

Any chance you can prod the maintainer of zeromq plugins to take a look at this?

I don't think these plugin have a maintainer.

12

Hi Bud,

yes I managed to get it to work.
Short informal description: Clone the logstash-mixin-zeromq "plugin" - build it locally and install. Then install the corresponding zeromq filter plugins (via the "standard" command that downloads the gem). I can look up more details (sorry don't have my notes at hand currently) if needed.

13

@burnard11 Can you do a git diff for each of the repos and attach the output? I'm assuming you needed to fix up the Gemfiles and maybe some other things... Thanks and Happy New Year! -- Bud

14

Hi Bud! Actually I did not fix it anything. Just building the zeromq-mixin against the current logstash was sufficient to resolve the version problem. That makes that whole issue so frustrating - basically everything is fine it is just the gem that is not uploaded in an up-to-date version to rubygems.

Happy new year!

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.