Can I use internal IP to connect to kafka? (logstash-input-kafka)

Elastic Stack Logstash
1

I have both my kafka and logstash in same region on GCP, separating machine. When I check the bandwidth usage of kafka's machine (iftop -P), I found that a number of bandwidth is from logstash machine's external ip. But I have set an internal ip on bootstrap_servers.

Now I want my logstash to connect to kafka using internal network. Is this supported? Or is it my settings' issue?

Say, x.x.x.x is external ip of my kafka machine, y.y.y.y is its internal ip. Some key kafka configuration i'm using: (kafka version; 0.11.0.2)

inter.broker.listener.name: INTERNAL
advertised.listeners: EXTERNAL://x.x.x.x:9092,INTERNAL://y.y.y.y:19092
listeners: EXTERNAL://0.0.0.0:9092,INTERNAL://0.0.0.0:19092
listener.security.protocol.map: PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT

And the logstash input configuration: (logstash version: 5.6.2)

input {
    kafka {
        topics => ["test"] 
        bootstrap_servers => "y.y.y.y:9092"
    }
}
2

Not with the current configured port, as per you config is internal port 19092 and not 9092

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.