Loop on list topic

Hi.
I have 30 topics kafka from server1 and I want to send all these topics to server2.
my input config is:

input { 
  kafka { 
        bootstrap_servers => "192.168.1.101:9092" 
        group_id => "all_topic" 
        topics => ["topic1, topic2,topic3,......,topic30"] 
        auto_offset_reset => "earliest" 
        decorate_events => true 
  } 
} 

I want every topic send to new tpoic name

output { 
   if ([@metadata][kafka][topic] == "topic1") { 
     kafka { 
       bootstrap_servers => "192.168.1.102:9092" 
       topic_id => "new_topic1" 
       codec => json 
          } 
if ([@metadata][kafka][topic] == "topic2") { 
     kafka { 
       bootstrap_servers => "192.168.1.102:9092" 
       topic_id => "new_topic2" 
       codec => json 
          } 
    } 
} 

Now, If I want to continue this way, The file gets too big for 30 topics, There is a way to apply to everyone in a block. A loop in the output block.
thanks.

If the only thing that changes is the topic, how about using a https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html filter to do the mapping? Then you can reference the translation in the kafka output

topic_id => "%{translation}"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.