# Logstash 5.2 not working with Kafka 9 and AVRO **URL:** \ **Category:** Logstash\ **Created:** [February 3, 2017, 8:28am UTC](https://discuss.elastic.co/t/logstash-5-2-not-working-with-kafka-9-and-avro/73810 "2017-02-03T08:28:41Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Zeeshan\_Alam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zeeshan_alam/32/94298_2.png) [@Zeeshan\_Alam](https://discuss.elastic.co/u/Zeeshan_Alam)\ **Post date:** [February 3, 2017, 8:28am UTC](https://discuss.elastic.co/t/logstash-5-2-not-working-with-kafka-9-and-avro/73810/1 "2017-02-03T08:28:41Z")
Logstash 5.2 not working with Kafka 9 and AVRO I have installed **Logstash 5.2.0** with **logstash-input-kafka 4.1.1** and **logstash-codec-avro 3.0.0** and trying to read data from **Cloudera Kafka 9** but I am getting the gollowing error: ``` [2017-02-03T03:05:35,049][INFO][logstash.pipeline] Pipeline main started [2017-02-03T03:05:35,064][DEBUG][logstash.agent] Starting puma [2017-02-03T03:05:35,065][DEBUG][logstash.agent] Trying to start WebServer {:port=>9600} [2017-02-03T03:05:35,068][DEBUG][logstash.api.service] [api-service] start [2017-02-03T03:05:35,090][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600} [2017-02-03T03:05:35,303][DEBUG][logstash.inputs.kafka] closing {:plugin=>"LogStash::Inputs::Kafka"} [2017-02-03T03:05:35,304][DEBUG][logstash.pipeline] Input plugins stopped! Will shutdown filter/output workers. [2017-02-03T03:05:35,338][DEBUG][logstash.pipeline] Pushing flush onto pipeline [2017-02-03T03:05:35,339][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-02-03T03:05:35,340][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-02-03T03:05:35,340][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-02-03T03:05:35,340][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-02-03T03:05:35,340][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-02-03T03:05:35,341][DEBUG][logstash.pipeline] Pushing shutdown {:thread=>"#"} [2017-02-03T03:05:35,341][DEBUG][logstash.pipeline] Shutdown waiting for worker thread # [2017-02-03T03:05:35,439][FATAL][logstash.runner] An unexpected error occurred! {:error=>#, :backtrace=>["org/jruby/ext/stringio/StringIO.java:829:in `read'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:106:in `read'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:93:in `read_bytes'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:304:in `read_data'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:290:in `read_data'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:376:in `read_union'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:309:in `read_data'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:384:in `read_record'", "org/jruby/RubyArray.java:1613:in `each'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:382:in `read_record'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:310:in `read_data'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/avro-1.8.1/lib/avro/io.rb:275:in `read'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/logstash-codec-avro-3.0.0-java/lib/logstash/codecs/avro.rb:73:in `decode'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-4.1.1/lib/logstash/inputs/kafka.rb:163:in `thread_runner'", "file:/apps/logstash-5.2.0/vendor/jruby/lib/jruby.jar!/jruby/java/java_ext/java.lang.rb:12:in `each'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-4.1.1/lib/logstash/inputs/kafka.rb:162:in `thread_runner'"]} [2017-02-03T03:05:35,444][DEBUG][logstash.agent] Error in reactor loop escaped: Bad file descriptor - Bad file descriptor (Errno::EBADF) [2017-02-03T03:05:35,445][DEBUG][logstash.agent] ["org/jruby/RubyIO.java:3705:in `select'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/puma-2.16.0-java/lib/puma/reactor.rb:29:in `run_internal'", "/apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/puma-2.16.0-java/lib/puma/reactor.rb:138:in `run_in_thread'"] [2017-02-03T03:05:35,445][DEBUG][logstash.agent] 2017-02-03 03:05:35 -0500: Listen loop error: # [2017-02-03T03:05:35,446][DEBUG][logstash.agent] org/jruby/RubyIO.java:3705:in `select' /apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/puma-2.16.0-java/lib/puma/server.rb:322:in `handle_servers' /apps/logstash-5.2.0/vendor/bundle/jruby/1.9/gems/puma-2.16.0-java/lib/puma/server.rb:296:in `run' [2017-02-03T03:05:35,446][DEBUG][logstash.agent] Error in reactor loop escaped: Bad file descriptor - Bad file descriptor (Errno::EBADF) ``` Same works well with **Logstash 2.4** and **logstash-codec-avro 2.0.4.** Here is my logstash config file: ``` input { kafka { bootstrap_servers => "server1:9092,server2:9092,server3:9092,server4:9092,server5:9092" topics => ["mytopicname"] group_id => "mygroup" codec => avro { schema_uri => "/apps/schema/rocana3.schema" } } } filter { } output { file { path => "/apps/elk/test/dump.txt" } } ``` ---
**Author:** ![Zeeshan\_Alam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zeeshan_alam/32/94298_2.png) [@Zeeshan\_Alam](https://discuss.elastic.co/u/Zeeshan_Alam)\ **Post date:** [February 8, 2017, 9:45am UTC](https://discuss.elastic.co/t/logstash-5-2-not-working-with-kafka-9-and-avro/73810/2 "2017-02-08T09:45:34Z")
The default serializer was changed in logstash 5 from byte array deserializer to string deserializer. I simply added the following to my input \> kafka and it worked. **key\_deserializer\_class** =\> "org.apache.kafka.common.serialization.ByteArrayDeserializer" **value\_deserializer\_class** =\> "org.apache.kafka.common.serialization.ByteArrayDeserializer" ``` input { kafka { bootstrap_servers => "server1:9092,server2:9092,server3:9092,server4:9092,server5:9092" topics => ["mytopicname"] group_id => "mygroup" key_deserializer_class => "org.apache.kafka.common.serialization.ByteArrayDeserializer" value_deserializer_class => "org.apache.kafka.common.serialization.ByteArrayDeserializer" codec => avro { schema_uri => "/apps/schema/rocana3.schema" } } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 8, 2017, 9:45am UTC](https://discuss.elastic.co/t/logstash-5-2-not-working-with-kafka-9-and-avro/73810/3 "2017-03-08T09:45:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.