Logstash-filter-zeromq does not install (version conflict) on logstash 5.1

Hi all,

I have trouble installing the logstah-filter-zeromq plugin on a clean logstash 5.1 install (tested with docker hub docker image logstash:5.1). Installation reports a version conflict. Full console output following below. The command used for installation is

logstash-plugin install logstash-filter-zeromq

Any advice is highly appreciated. Thanks in advance!

Console output [truncated due to message size limit]:

   Validating logstash-filter-zeromq
Installing logstash-filter-zeromq
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core":
  In snapshot (Gemfile.lock):
    logstash-core (= 5.1.1)

  In Gemfile:
    logstash-core-plugin-api (>= 0) java depends on
      logstash-core (= 5.1.1) java

    logstash-filter-zeromq (>= 0) java depends on
      logstash-core (< 2.0.0, >= 1.4.0) java

    logstash-core (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "logstash-core-plugin-api":
  In snapshot (Gemfile.lock):
    logstash-core-plugin-api (= 2.1.12)

  In Gemfile:
    logstash-devutils (>= 0) java depends on
      logstash-core-plugin-api (~> 2.0) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

   [...]

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-output-file (>= 0) java depends on
      logstash-core-plugin-api (< 2.99, >= 2.0.0) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-output-stdout (>= 0) java depends on
      logstash-core-plugin-api (< 2.99, >= 1.60.1) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-filter-zeromq (>= 0) java depends on
      logstash-core-plugin-api (~> 1.0) java

    logstash-core-plugin-api (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "logstash":
  In Gemfile:
    logstash-filter-zeromq (>= 0) java depends on
      logstash (< 2.0.0, >= 1.4.0) java
Could not find gem 'logstash (< 2.0.0, >= 1.4.0) java', which is required by gem 'logstash-filter-zeromq (>= 0) java', in any of the sources.
ERROR: Service 'logstash' failed to build: The command '/bin/sh -c logstash-plugin install logstash-filter-zeromq' returned a non-zero code: 1

The issue is not docker specific but also happens on a clean Ubuntu 16.04 (using opejdk-8 jre). Any advice is highly appreciated!
Alternatively: Is there any other method recommended to send logstash events to an external service for filtering?

Now also tried to build the plugin locally and get a more concise error message. It seems that the core-plugin-api and the zeromq mixin seem to require different versions of logstash-core. Is there any way to work aorund this?

ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core":
In snapshot (Gemfile.lock):
  logstash-core (= 5.1.1)

In Gemfile:
  logstash-core-plugin-api (>= 0) java depends on
    logstash-core (= 5.1.1) java

  logstash-filter-zeromq (= 3.0.0) java depends on
    logstash-mixin-zeromq (>= 2.0.1) java depends on
      logstash-core (< 3.0.0, >= 2.0.0) java

  logstash-core (>= 0) java

It looks like the zeromq plugin isn't compatible with Logstash 5.

Thanks for the answer! Is this really an incompatibility in terms of function?
Because from what I can judge the necessary logstash-mixin-zeromq is uploaded to rubygems.org only in version 2.0.2.
When I download the logstash-mixin-zeromq locally it is built succesfully and generates a version 3.0.0 with a dependency on logstash core that is suitbale for 5.11, and therefore can be installed.
I am currently struggling to get everything installed in my local logstash so can not (yet) make any assumptions if it will work properly

It could very well be that the maintainer of that plugin only needs to make a new release.

For further reference: The issue seems to known and is tracked on github at https://github.com/logstash-plugins/logstash-mixin-zeromq/issues/8

Final comment: I can confirm now that the problem is only caused by an outdated logstash-mixin-zeromq which is not "build against" the current logstash version.

@magnusbaeck Any chance you can prod the maintainer of zeromq plugins to take a look at this? -- Bud

Daniel, Were you able to build this on your own to get it to work? If so, can you provide a recipe? -- Bud

Any chance you can prod the maintainer of zeromq plugins to take a look at this?

I don't think these plugin have a maintainer.

Hi Bud,

yes I managed to get it to work.
Short informal description: Clone the logstash-mixin-zeromq "plugin" - build it locally and install. Then install the corresponding zeromq filter plugins (via the "standard" command that downloads the gem). I can look up more details (sorry don't have my notes at hand currently) if needed.

@burnard11 Can you do a git diff for each of the repos and attach the output? I'm assuming you needed to fix up the Gemfiles and maybe some other things... Thanks and Happy New Year! -- Bud

Hi Bud! Actually I did not fix it anything. Just building the zeromq-mixin against the current logstash was sufficient to resolve the version problem. That makes that whole issue so frustrating - basically everything is fine it is just the gem that is not uploaded in an up-to-date version to rubygems.

Happy new year!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.