# Kafka input with different deserializer classes

**URL:** <https://discuss.elastic.co/t/kafka-input-with-different-deserializer-classes/127352>\
**Category:** Logstash\
**Created:** [April 9, 2018, 3:46pm UTC](https://discuss.elastic.co/t/kafka-input-with-different-deserializer-classes/127352 "2018-04-09T15:46:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![guidoilbaldo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guidoilbaldo/32/44445_2.png) [@guidoilbaldo](https://discuss.elastic.co/u/guidoilbaldo)\
**Post date:** [April 9, 2018, 3:46pm UTC](https://discuss.elastic.co/t/kafka-input-with-different-deserializer-classes/127352/1 "2018-04-09T15:46:32Z")

</div>

Hi,

I'm trying to consume events from Kafka with Logstash to send them to Elastic.  
I'm currently using a pipeline with the following input:

```
  kafka {
    type => "standard_event"
    bootstrap_servers => "xxx01:9092,xxx02:9092,xxx03:9092,xxx04:9092"
    topics => ["something"]
    group_id => "somehow"
    decorate_events => true
    consumer_threads => 10
    key_deserializer_class => "org.apache.kafka.connect.storage.StringConverter"
    value_deserializer_class => "io.confluent.connect.avro.AvroConverter"
    codec => avro {
      schema_uri => "/root/schema.avsc"
      tag_on_failure => true
    }
  }

```

I'm trying to use different classes in order to deserialize events.  
I put those JARs under /usr/share/logstash/lib/ but, somehow, they are not found by Logstash when I launch it.  
I get the following error:

```
Error: uncaught throw org.apache.kafka.common.config.ConfigException: Invalid value org.apache.kafka.connect.storage.StringConverter for configuration key.deserializer: Class org.apache.kafka.connect.storage.StringConverter could not be found.
  Exception: UncaughtThrowError
  Stack: org/jruby/RubyKernel.java:1137:in `throw'

```

Does anyone know where should I put those JARs so that Logstash can use them?

---

<div class="post-metadata">

**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\
**Post date:** [April 10, 2018, 12:57am UTC](https://discuss.elastic.co/t/kafka-input-with-different-deserializer-classes/127352/2 "2018-04-10T00:57:53Z")

</div>

This shouldn't be throwing an `UncaughtThrowError`, and I've opened [logstash-plugins/logstash-input-kafka#259](https://github.com/logstash-plugins/logstash-input-kafka/pull/259) to address this.

However, even after the above fix gets merged, you'll still have a problem; while this plugin allows us to specify the _name_ of a class, it does not do anything to add jars to the class loader, and expects the classes being referenced to already exist on the class path.

It looks like there's an open ticket to add this functionality: [logstash-plugins/logstash-input-kafka#238](https://github.com/logstash-plugins/logstash-input-kafka/issues/238); feel free to subscribe and/or contribute 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 8, 2018, 12:58am UTC](https://discuss.elastic.co/t/kafka-input-with-different-deserializer-classes/127352/3 "2018-05-08T00:58:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.