Kafka plugin try to describe all the topics when used `topics_pattern`

Elastic Stack Logstash
1

Hi,

We are using Logstash Kafka input plugin which is connecting to Kafka with below configuration.

  kafka {
    codec => "json"
    bootstrap_servers => "1.2.3.4:9094"
    topics_pattern => "^something-.*$"
    group_id => "something-group"
    consumer_threads => 4
    client_id => "something-group"
    decorate_events => true
    auto_offset_reset => "earliest"
    security_protocol => "SSL"
    ssl_key_password => "${USERTRUSTSTOREPASS}"
    ssl_keystore_location => "/usr/share/logstash/truststore/user.truststore"
    ssl_keystore_password => "${USERTRUSTSTOREPASS}"
    ssl_truststore_type => "jks"
    ssl_truststore_location => "/usr/share/logstash/truststore/server.truststore"
    ssl_truststore_password => "${SERVERTRUSTSTOREPASS}"
    ssl_truststore_type => "jks"
  }

The Kafka is running with PREFIXED ACL security and above credentials have something-* access.

The above logstash are trying to run Describe on all the topics in Kafka instead of just the index patterns one. I believe this isn't desired? Let me know if I should create bug for it in logstash-kafka-input repository.

2019-03-28 14:42:35,217 INFO Principal = User:CN=something-group is Denied Operation = Describe from host = 172.16.6.0 on resource = Topic:LITERAL:some-other-topic (kafka.authorizer.logger) [kafka-request-handler-3]

Though it isn't causing performance issue right now BUT the failed ACLs attempt detection is getting spammed due to the same.

Kafka Version: 2.0.1
Logstash Version tried with: 6.3.2 & 6.6.2

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.