# Add custom encoder for kafka output plugin

**URL:** <https://discuss.elastic.co/t/add-custom-encoder-for-kafka-output-plugin/82655>\
**Category:** Logstash\
**Created:** [April 18, 2017, 4:24am UTC](https://discuss.elastic.co/t/add-custom-encoder-for-kafka-output-plugin/82655 "2017-04-18T04:24:10Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![codenamelxl](https://avatars.discourse-cdn.com/v4/letter/c/ad7895/32.png) [@codenamelxl](https://discuss.elastic.co/u/codenamelxl)\
**Post date:** [April 18, 2017, 4:24am UTC](https://discuss.elastic.co/t/add-custom-encoder-for-kafka-output-plugin/82655/1 "2017-04-18T04:24:10Z")

</div>

After searching around and reading source of kafka output plugin. I think that i can support avro encoding with Confluent schema registry format by adding some config to the plugin and use io.confluent.kafka.serializers.KafkaAvroEncoder as encoder class.  
However, i can't find any way to add relevant jar files so that jruby can find it. I tried to pack it to the gem under /vendor/jar-dependecies/runtime-jars but logstash still complaint that it cant found the needed classes.

[2017-04-18T04:22:35,187][ERROR][logstash.agent] Pipeline aborted due to error {:exception=\>#\<NameError: cannot link Java class org.apache.kafka.clients.producer.ProducerConfig (java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory)\>, :backtrace=\>["org/jruby/javasupport/JavaUtilities.java:54:in `get_proxy_or_package_under_package'", "file:/usr/share/logstash/vendor/jruby/lib/jruby.jar!/jruby/java/java_package_module_template.rb:14:in`method\_missing'", "/usr/share/logstash/vendor/local\_gems/2a463982/logstash-output-kafka-6.1.5/lib/logstash/outputs/kafka.rb:218:in `create_producer'", "/usr/share/logstash/vendor/local_gems/2a463982/logstash-output-kafka-6.1.5/lib/logstash/outputs/kafka.rb:184:in`register'", "/usr/share/logstash/logstash-core/lib/logstash/output\_delegator\_strategies/shared.rb:8:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:37:in`register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:282:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:293:in`register\_plugins'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:293:in`register\_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:302:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:232:in`run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:387:in `start\_pipeline'"]}

Also, adding the jars into CLASSPATH does not help.  
I cant find any document on how to use java files with logstash plugin. Most things i found are from 1-2 years ago.  
Edit: my jars are not avaiable on maven

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 16, 2017, 4:30am UTC](https://discuss.elastic.co/t/add-custom-encoder-for-kafka-output-plugin/82655/2 "2017-05-16T04:30:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.