When I use kafka plugin in logstash, I am getting the below error.
[2023-03-10T15:11:46,310][WARN ][org.apache.kafka.clients.NetworkClient][main][289f84d5c44d64af9505535a5352178af25a608c83252a1c520ab39a4faa4855] [Consumer clientId=logstash-0, groupId=logstash] Error connecting to node kafka-broker:9092 (id: 1 rack: null)
Yaml file :
bootstrap_servers => "localhost:9092"
topics => ["topicdemo"]
auto_offset_reset => "earliest"
decorate_events => true
hosts => ["https://localhost:9200"]
cacert => 'C:\logstash-8.6.2-windows-x86_64\logstash-8.6.2\config\http-cert\ca.crt'
index => "topicdemo"
user => "elastic"
password => "********"
Looks like the program can't find the host called kafka-broker, but in your yaml file it is trying to bootstrap kafka on localhost right? Maybe try switching from kafka-broker to localhost? Just a guess.
I do not have kafka running these days, but it could be that the list of hosts for that topic in the bootstrap server has a hostname that logstash cannot resolve. Perhaps the bootstrap server should be configured to return an FQDN.
What is the
advertised.listeners of your Kafka Cluster?
bootstrap_servers in Logstash will get the the list of host from the configured server and this list will be the configured
advertised.listeners, it is probably related to what @Badger mentioned.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.