Kafka output plugin java.lang.long error

When I try to use the Kafka output plugin I get this variable type error:

[ERROR][logstash.outputs.kafka ] Unable to create Kafka producer from given configuration {:kafka_error_message=>org.apache.kafka.common.config.ConfigException: Invalid value 32768 for configuration receive.buffer.bytes: Expected value to be a 32-bit integer, but it was a java.lang.Long, :cause=>nil}

I am just testing, so the output configuration is very simple:

  kafka {
    codec => json
    topic_id => "logstash-test"
    bootstrap_servers => "foo.example.com:9092"

Logstash version is 6.3.0, and Kafka output plugin version is 7.1.2. The platform is CentOS6 with Java OpenJDK version 1.8.0_171.

I have tried explicitly setting the receive_buffer_bytes parameter, with and without enclosing in quotes, but that made no difference. Does anybody have any ideas on how to fix please?

For the benefit of anyone else with this issue, upgrading Logstash to v6.3.2 resolved this issue.

FYI I am using Elastic Official Docker Images. I updated the Logstash Image 6.3.2 to 6.4.0 and the issue is the same. The kafka output plugin 7.1.2 seems to be the issue. https://github.com/logstash-plugins/logstash-output-kafka/pull/198 . FYI Version ELK 6.3.2 was working fine.

Logstash 6.3.2 was released on July 24th (https://www.elastic.co/downloads/past-releases) and the Logstash Kafka Output Plugin 7.1.1 was released on June 5th (https://www.elastic.co/guide/en/logstash-versioned-plugins/current/v7.1.1-plugins-outputs-kafka.html). I am using the official Docker Image for Logstash and the 6.3.2 then must have been shipped with the Logstash Kafka Output Plugin version 7.1.1 . After updating to Docker Image 6.4.0 (released August 23rd,2018) the Logstash Kafka Output Plugin Version 7.1.2 (released August 8th, 2018) is released as part of the image. I rolled back the Kafka output plugin the following way:

bin/logstash-plugin uninstall logstash-output-kafka
bin/logstash-plugin install --version 7.1.1 logstash-output-kafka

After doing that, everything now works fine

if you want to remove version 7.1.2 and install 7.1.1 . while Logstash is starting for the firs time you can do the following:

logstash-plugin remove logstash-output-kafka
logstash-plugin install --version 7.1.1 logstash-output-kafka

you should then get the following before running logstash:

Successfully removed logstash-output-kafka
Validating logstash-output-kafka-7.1.1
Installing logstash-output-kafka
Installation successful
1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.