# Logstash: dynamic topic based on value of field

**URL:** <https://discuss.elastic.co/t/logstash-dynamic-topic-based-on-value-of-field/310976>\
**Category:** Logstash\
**Created:** [July 29, 2022, 10:28am UTC](https://discuss.elastic.co/t/logstash-dynamic-topic-based-on-value-of-field/310976 "2022-07-29T10:28:11Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jori-be](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jori-be/32/96028_2.png) [@jori-be](https://discuss.elastic.co/u/jori-be)\
**Post date:** [July 29, 2022, 10:28am UTC](https://discuss.elastic.co/t/logstash-dynamic-topic-based-on-value-of-field/310976/1 "2022-07-29T10:28:11Z")

</div>

I'm trying to configure Logstash with Kafka output. Basically I want to put data in different Kafka topics based on a field.  
So in example below, I have a field name in the data. All documents should be posted to the topic per name.  
E.g. if name is test1, topic should be topic\_test1, for test2 it should be topic\_test2 etc...

```auto
output {
  kafka {
    id => "topic-kafka"
    bootstrap_servers => "kafka.test.be:9092"
    topic_id => "topic_%{name}"
    codec => json
  }
}

```

I tried to use the sprintf format like above, but that doesn't work.

> **[Accessing event data and fields | Logstash Reference \[8.3\] | Elastic](https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#sprintf)**

`[2022-07-29T11:51:52,502][ERROR][org.apache.kafka.clients.Metadata][topic-kafka] [Producer clientId=producer-1] Metadata response reported invalid topics [topic_%{name}]`

I'm using logstash version v7.16.2

Any ideas how we can configure this?

Thanks!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [July 29, 2022, 11:54am UTC](https://discuss.elastic.co/t/logstash-dynamic-topic-based-on-value-of-field/310976/2 "2022-07-29T11:54:26Z")

</div>

This is supported and should work if you have the `name` field in your document in logstash.

Can you share what is the output in logstash when you got that error? You can add an extra `stdout` output to see the document that logstash is sending.

---

<div class="post-metadata">

**Author:** ![jori-be](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jori-be/32/96028_2.png) [@jori-be](https://discuss.elastic.co/u/jori-be)\
**Post date:** [August 1, 2022, 9:10am UTC](https://discuss.elastic.co/t/logstash-dynamic-topic-based-on-value-of-field/310976/3 "2022-08-01T09:10:27Z")

</div>

> [@jori-be](#):
>
> `%{name}`

Indeed, there was a mistake in my logstash. This caused that the name field was not existing, therefore the error was popping up.  
Thanks to point that out.  
So the output config as I posted was correct after all.

Thanks, ticket can be closed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 29, 2022, 9:10am UTC](https://discuss.elastic.co/t/logstash-dynamic-topic-based-on-value-of-field/310976/4 "2022-08-29T09:10:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.