# Multiple kafka topics using logstash and make index inside output with \`.conf\` file

**URL:** <https://discuss.elastic.co/t/multiple-kafka-topics-using-logstash-and-make-index-inside-output-with-conf-file/352112>\
**Category:** Logstash\
**Created:** [January 30, 2024, 5:45pm UTC](https://discuss.elastic.co/t/multiple-kafka-topics-using-logstash-and-make-index-inside-output-with-conf-file/352112 "2024-01-30T17:45:01Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tony\_Haf.H](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tony_haf.h/32/125662_2.png) [@Tony\_Haf.H](https://discuss.elastic.co/u/Tony_Haf.H)\
**Post date:** [January 30, 2024, 5:45pm UTC](https://discuss.elastic.co/t/multiple-kafka-topics-using-logstash-and-make-index-inside-output-with-conf-file/352112/1 "2024-01-30T17:45:01Z")

</div>

I am using Logstash 8.11, and I have problem like subject title

I have consulted a few solutions in other topics, and I still have not solved the problem.  
Example old topic: [How to pull data data from 2 kafka topics using logstash and index the data in two separate index in elasticsearch](https://discuss.elastic.co/t/how-to-pull-data-data-from-2-kafka-topics-using-logstash-and-index-the-data-in-two-separate-index-in-elasticsearch/114977/1) with same case for resolve by [guyboertje](https://discuss.elastic.co/u/guyboertje), but it not working now

a few other related issues but using with Filebeat, but I'm not like that, I'm using according to ".conf" file

This is inside my ".conf" file

```auto
input {
  kafka {
    bootstrap_servers => "localhost:9092"
    topics => ["a_system_logs", "b_system_logs"]
  }
}
filter {
  mutate {
    add_field => { "es_index" => "logstash_%{[@metadata][kafka][topic]}" }
  }
}	
output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "%{es_index}"
  }
}

```

It not working. When service send data, it will show:

`[2024-01-31T00:32:36,736][WARN][logstash.outputs.elasticsearch][main][...] Badly formatted index, after interpolation still contains placeholder: [logstash_%{[@metadata][kafka][topic]}]; event: `{"@timestamp"=>2024-01-30T17:32:36.569868Z, "@version"=>"1", "es_index"=>"logstash_%{[@metadata][kafka][topic]}", "message"=>" ***my data***"`

Is there anyone using ".conf" file, please show me how to solve the problem?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [January 30, 2024, 6:24pm UTC](https://discuss.elastic.co/t/multiple-kafka-topics-using-logstash-and-make-index-inside-output-with-conf-file/352112/2 "2024-01-30T18:24:59Z")

</div>

Metadata is not added to events by default. You need to set the [decorate\_events option](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-kafka.html#_metadata_fields).

---

<div class="post-metadata">

**Author:** ![Tony\_Haf.H](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tony_haf.h/32/125662_2.png) [@Tony\_Haf.H](https://discuss.elastic.co/u/Tony_Haf.H)\
**Post date:** [January 30, 2024, 6:58pm UTC](https://discuss.elastic.co/t/multiple-kafka-topics-using-logstash-and-make-index-inside-output-with-conf-file/352112/3 "2024-01-30T18:58:09Z")

</div>

I can't believe such a small installation step is missing.  
I looked some other topics, they didn't even have this step, and I didn't pay attention.  
Thanks so much.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 27, 2024, 6:58pm UTC](https://discuss.elastic.co/t/multiple-kafka-topics-using-logstash-and-make-index-inside-output-with-conf-file/352112/4 "2024-02-27T18:58:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.