Logstash Avro Kafka Codec issue

Elastic Stack Logstash
1

Hi all,

I am running a simple logstash pipeline to consume a message from a Kafka topic, which is in AVRO format and display it i on the stdout. I get all empty elements in the logstash output.

AVRO Schema (bestMatchingOutput.avsc)

{"namespace":"com.test.bestmatcher.avro","type":"record","name":"AvroBestMatchingOutput","fields":[{"name":"orgId","type":"string"},{"name":"userId","type":"string"},{"name":"matchId","type":"string"},{"name":"matcherVersion","type":"string"}]}

AVRO message in Kafka published through Kafka Avro console Producer. I can easily consume this message using Kafka Avro console consumer.

{"orgId":"xyz","userId":"1595abbd3c6a22883814","matchId":"296EAADC0149A00D7","matcherVersion":"2.0.3"}

Logstash template

    input{
        kafka {
        codec => avro {
            schema_uri => "/testing/samplemessages/bestMatchingOutput.avsc"
        }
        key_deserializer_class => "org.apache.kafka.common.serialization.ByteArrayDeserializer"
        value_deserializer_class => "org.apache.kafka.common.serialization.ByteArrayDeserializer"
        bootstrap_servers => "localhost:9092"
        topics => ["topic1","topic2","topic3","topic4","topic5"]         
        decorate_events => "true"
      }
    }
filter {}

output{
    stdout { 
          codec => rubydebug 
          }      
}

No matter what I do the logstash output is empty as shown below

{
            "matchId" => "",
             "orgId" => "",
            "userId" => "",
        "@timestamp" => 2020-02-02T04:59:28.366Z,
    "matcherVersion" => "",
          "@version" => "1"
}

Any help is greatly appreciated. I am truly stuck with this issues.

Thanks
Nick

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.