Logstash plugin fails to run from source


First, please note that I'm pretty new to Ruby.

I would like to play around with the logstash-output-kafka plugin for educational purposes, but I can't get it to run even before I have made any modifications to it.

What I have done:

  1. Installed Logstash (version 6.5.4)
  2. Update the existing plugin (logstash-plugin update logstash-output-kafka)
  3. Check existing version (logstash-plugin list --verbose | grep kafka)
  4. Verify it is working by starting Logstash
  5. Clone source code of the same version
  6. Build the gem (gem build logstash-output-kafka.gemspec)
  7. Install gem (logstash-plugin install logstash-output-kafka-8.0.0.gem)
  8. Start Logstash

But it fails with
Couldn't find any output plugin named 'kafka'. Are you sure this is correct? Trying to load the kafka output plugin resulted in this error: Problems loading the requested plugin named kafka of type output. Error: RuntimeError you might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars withlock_jarscommand\n\nno such file to load -- org/apache/kafka/kafka-clients/2.1.0/kafka-clients-2.1.0 (LoadError)

I have tried to install dependencies according to the instructions on the Github page (https://github.com/logstash-plugins/logstash-output-kafka). I have tried with the JRuby binary used by Logstash as well as a freshly downloaded.
Logstash always fails with the same error.

I have found lots of people reporting the same issue but no one seems to share an answer to what is missing.

What am I doing wrong?

Best regards

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.