Logstash-input-kafka modify non-ascii

Elastic Stack Logstash
1

Hi,

I meet a problem when using logstash-input-kafka plugin recently.
I write a serialized protobuf message to kafka, and then use logstash-input-kafka plugin to consume the messages in logstash, but I find that some none-ascii chars have been modified like: 0x0aeb => 0x0aef 0xbfdb, which makes parsing protobuf message from string failed, here is the version details:

kafka: 2.10-0.10.0.0
logstash: 5.2.2
logstash-plugin-kafka: 5.1.6

Input part of logstash configuration:

input {
kafka {
bootstrap_servers => “host:9092"
topics => ’top_name'
codec => protobuf
{
class_name => “Module::ClassName"
include_path => [‘/path/ClassName.pb.rb']
}
}
}

Is this a bug of logstash-input-kafka plugin, or does I miss some configurations?

2

Problem solved by adding:

value_deserializer_class => "org.apache.kafka.common.serialization.ByteArrayDeserializer"

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.