Having some struggles getting kafka-input setup in logstash with SASL PLAINTEXT. Documentation seems to be largely non-existent. I did seem to get Beats sending data in to Kafka with this auth scheme.
The input is pretty basic:
input {
kafka {
bootstrap_servers => "10.99.250.74:9092"
topics => ["beats-llk"]
sasl_jaas_config => "org.apache.kafka.common.security.plain.PlainLoginModule required username='admin' password='SOMEPASS';"
sasl_mechanism => "PLAIN"
group_id => "logstash-llk"
}
}
I have tried using a specific user for logstash that limits access via ACL to just the topics we want it to touch and that was failing so I opened it up to the main admin/superuser password just to try and eliminate ACL issues. In the kafka server I am seeing the following messages stream out:
[2020-02-15 08:31:44,680] INFO [SocketServer brokerId=1] Failed authentication with /10.99.242.21 (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
Any suggestions or pointers to something better than super basic docs would be appreciated.