I have both my kafka and logstash in same region on GCP, separating machine. When I check the bandwidth usage of kafka's machine (iftop -P), I found that a number of bandwidth is from logstash machine's external ip. But I have set an internal ip on bootstrap_servers.
Now I want my logstash to connect to kafka using internal network. Is this supported? Or is it my settings' issue?
Say, x.x.x.x is external ip of my kafka machine, y.y.y.y is its internal ip. Some key kafka configuration i'm using: (kafka version; 0.11.0.2)
inter.broker.listener.name: INTERNAL
advertised.listeners: EXTERNAL://x.x.x.x:9092,INTERNAL://y.y.y.y:19092
listeners: EXTERNAL://0.0.0.0:9092,INTERNAL://0.0.0.0:19092
listener.security.protocol.map: PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT,INTERNAL:PLAINTEXT
And the logstash input configuration: (logstash version: 5.6.2)
input {
kafka {
topics => ["test"]
bootstrap_servers => "y.y.y.y:9092"
}
}