# Add array items as prefixes for another array

**URL:** <https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134>\
**Category:** Logstash\
**Created:** [February 22, 2018, 9:40pm UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134 "2018-02-22T21:40:29Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Maher\_Ben\_Taleb\_Ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maher_ben_taleb_ali/32/26774_2.png) [@Maher\_Ben\_Taleb\_Ali](https://discuss.elastic.co/u/Maher_Ben_Taleb_Ali)\
**Post date:** [February 22, 2018, 9:40pm UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134/1 "2018-02-22T21:40:29Z")

</div>

Hi,  
I am trying to push articles into the corresponding Kafka Topics.

```
article : {
 "topics": [
      "Barclays",
      "Companies",
      "Government",
      "Warrant"
    ],
    "categories": [
      "economy",
      "business",
      "finance"
    ]
}

```

the corresponding kafka topics are in the form of Category\_Topics example:

["economy\_Barclays","business\_Barclays","finance\_Barclays","economy\_Government","business\_Government","finance\_Government"]

is this possible without using a Ruby filter ?

---

<div class="post-metadata">

**Author:** ![Maher\_Ben\_Taleb\_Ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maher_ben_taleb_ali/32/26774_2.png) [@Maher\_Ben\_Taleb\_Ali](https://discuss.elastic.co/u/Maher_Ben_Taleb_Ali)\
**Post date:** [February 23, 2018, 7:50pm UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134/2 "2018-02-23T19:50:35Z")

</div>

I used a Ruby Filter, and it is working correctly

```
input {
  elasticsearch {
    hosts => "localhost"
    index => "articles"
    add_field => { "kafka_topics" => '' }
  }
}

filter {
  ruby {
    code => "
	kafka_topics=[]
	categories=event.get('categories' )
	tpics=event.get('topics' )
	categories.each {|category| tpics.each{|topic| kafka_topics.push(category +'_'+ topic)} }
	event.set('kafka_topics', kafka_topics)
	"
  }
}
output {
  kafka {
    codec => json
    topic_id => "foo"
  }
}

```

My question here is there a way to push the event to multiple topics in Kafka.  
exatly like the input plugin for kafka?

---

<div class="post-metadata">

**Author:** ![Ranjith\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ranjith_m/32/19272_2.png) [@Ranjith\_M](https://discuss.elastic.co/u/Ranjith_M)\
**Post date:** [February 23, 2018, 8:18pm UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134/3 "2018-02-23T20:18:01Z")

</div>

Well only possible way which I can see is to pass a dynamic value to **topic\_id**

---

<div class="post-metadata">

**Author:** ![Maher\_Ben\_Taleb\_Ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maher_ben_taleb_ali/32/26774_2.png) [@Maher\_Ben\_Taleb\_Ali](https://discuss.elastic.co/u/Maher_Ben_Taleb_Ali)\
**Post date:** [February 23, 2018, 8:30pm UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134/4 "2018-02-23T20:30:48Z")

</div>

yes I am aware of dynamic values. the problem is that I don't have a predefined list of outputs or the number of outputs at least.  
is there a way to execute a ruby code in the output?

---

<div class="post-metadata">

**Author:** ![Ranjith\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ranjith_m/32/19272_2.png) [@Ranjith\_M](https://discuss.elastic.co/u/Ranjith_M)\
**Post date:** [February 23, 2018, 8:36pm UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134/5 "2018-02-23T20:36:27Z")

</div>

I don't think that is possible , an alternate way to restrict is provide a list and use them using Prune filter.  
Read through if that can work

---

<div class="post-metadata">

**Author:** ![Maher\_Ben\_Taleb\_Ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maher_ben_taleb_ali/32/26774_2.png) [@Maher\_Ben\_Taleb\_Ali](https://discuss.elastic.co/u/Maher_Ben_Taleb_Ali)\
**Post date:** [February 23, 2018, 8:49pm UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134/6 "2018-02-23T20:49:30Z")

</div>

does logstash supports partial config files?  
can't I run a ruby filter and write the outputs in a config file then upload it.  
does logstash loads the config before running?

---

<div class="post-metadata">

**Author:** ![Maher\_Ben\_Taleb\_Ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maher_ben_taleb_ali/32/26774_2.png) [@Maher\_Ben\_Taleb\_Ali](https://discuss.elastic.co/u/Maher_Ben_Taleb_Ali)\
**Post date:** [February 24, 2018, 1:33am UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134/7 "2018-02-24T01:33:47Z")

</div>

I forked the logstash-output-kafka, it is now supporting the multi topics output

```
  kafka {
    topic_ids => ["foo","test"]
  }

```

> **[MaherBTA/logstash-output-kafka](https://github.com/MaherBTA/logstash-output-kafka)**
>
> logstash-output-kafka - Kafka Output for Logstash

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 24, 2018, 1:33am UTC](https://discuss.elastic.co/t/add-array-items-as-prefixes-for-another-array/121134/8 "2018-03-24T01:33:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.