With reference to "Modern Kafka clients do not connect to Zookeeper. Historical versions of Kafka exposed Zookeeper to clients to retrieve the full broker list (Kafka 0.7) and the initial consumer groups were implemented in the clients using Zookeeper (Kafka 0.8). But since Kafka 0.8 (and 0.9 for consumer groups) there is no longer any need for clients to connect to Zookeeper, the metadata and consumer group functionality has moved to the Kafka broker. In fact, it is highly recommended not to make Zookeeper reachable from clients due to security aspects."
Tried to configure Logstash to take input from Kafka Cluster using bootstrap_servers option as follows:
input {
kafka {
topic_id => "ericsson"
bootstrap_servers => "192.168.4.79:9092"
type => "kafka-input"
}
}
which resulted in the following error
[root@elk ~]# /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf
Unknown setting 'bootstrap_servers' for kafka {:level=>:error}
Error: Something is wrong with your configuration. {:level=>:error}
[root@elk ~]#
When the same thing is run with Zoo Keeper option as follows it does work well
input {
kafka {
topic_id => "ericsson"
zk_connect => "192.168.4.79:2181"
type => "kafka-input"
}
}
Please provide your inputs
Thanks and Regards,
Rajendra