Error connecting to node kafka-broker:9092 (id: 1 rack: null) java.net.UnknownHostException: kafka-broker

[Rajesh_R]

When I use kafka plugin in logstash, I am getting the below error.

[2023-03-10T15:11:46,310][WARN ][org.apache.kafka.clients.NetworkClient][main][289f84d5c44d64af9505535a5352178af25a608c83252a1c520ab39a4faa4855] [Consumer clientId=logstash-0, groupId=logstash] Error connecting to node kafka-broker:9092 (id: 1 rack: null)
java.net.UnknownHostException: kafka-broker

Yaml file :
input {
kafka {
bootstrap_servers => "localhost:9092"
topics => ["topicdemo"]
auto_offset_reset => "earliest"
decorate_events => true
}
}
filter {

	}

output {
elasticsearch {
hosts => ["https://localhost:9200"]
cacert => 'C:\logstash-8.6.2-windows-x86_64\logstash-8.6.2\config\http-cert\ca.crt'
index => "topicdemo"
user => "elastic"
password => "********"
}
}

[Wave (Andrew Sullivan)]

Looks like the program can't find the host called kafka-broker, but in your yaml file it is trying to bootstrap kafka on localhost right? Maybe try switching from kafka-broker to localhost? Just a guess.

[Badger]

I do not have kafka running these days, but it could be that the list of hosts for that topic in the bootstrap server has a hostname that logstash cannot resolve. Perhaps the bootstrap server should be configured to return an FQDN.

[leandrojmp (Leandro Pereira)]

@Rajesh_R

Error connecting to node kafka-broker:9092 (id: 1 rack: null)
java.net.UnknownHostException: kafka-broker

What is the advertised.listeners of your Kafka Cluster?

The bootstrap_servers in Logstash will get the the list of host from the configured server and this list will be the configured advertised.listeners, it is probably related to what @Badger mentioned.