All our other kafka 0.9 clients work fine with 0.10 brokers. But whenever I start a kafka-output logstash hangs, and can only be recovered with kill -9. Nothing in the logs. (Have not run with -debug yet.)
kafka-input does not seem to have the same effect. But I don't really know if it's working at all as no input can send logs.
Just noticed that I have v 2.x of the plugins install. Seems I should have 3.x. Trying that now...
Nope, same behavior with 3.x plugins.
But I do know the plugin is starting, from logstash.err:
log4j:WARN No appenders could be found for logger (org.apache.kafka.clients.producer.ProducerConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
The logstash team put together a nice compatibility matrix on the logstash plugin page
Do you see any connections from logstash to kafka when running something like netstat?
I had two problems, both user error.
The first was the EC2 security groups did not allow egress to kafka. The second, I was using the docs released docs, not the master docs. It seems that 3.x has unified the plugin parameters between input and output.
Just a few minutes ago we finished updating the production environment from rabbit to kafka and all is good.