Pulling topic metadata from kafka 0.9

Elastic Stack Logstash
1

I am using logstash-input-kafka 3.0.x and logstash-output-kafka 3.0.x with logstash 2.2.2

How can I pull topic metadata from it. I could with zk_connect with plugin 2.x, but not with the plugin upgrade.

2

With kafka 2.x plugin using zk_connect I could run something like this

grok {
match => { "[kafka][topic]" => "%{WORD:network_hardware}-%{WORD}-%{WORD:dc}" }
}

where based on the kafka topic name, I could execute different cases.

My current input setup

kafka {
bootstrap_servers => "broker-0.blah.consul:7000,broker-1.blah.consul:7000,broker-2.blah.consul:7000"

bootstrap_servers => "test1.blah.com:2181"

      topics => ["f5test-logs-ev1","firewalltest-logs-ev1","paloaltotest-logs-ev1"]
      topics => ["f5-log-ev1","firewall-logs-ev1","paloalto-logs-ev1"]

      enable_auto_commit => "true"
      codec => plain
     }
3

This question has been answered by this PR. I believe it is designed for 0.10 but LS team is planning to port it to 0.9