# Does logstash can automatically consume new kafka topic?

**URL:** <https://discuss.elastic.co/t/does-logstash-can-automatically-consume-new-kafka-topic/283149>\
**Category:** Logstash\
**Created:** [September 2, 2021, 9:24am UTC](https://discuss.elastic.co/t/does-logstash-can-automatically-consume-new-kafka-topic/283149 "2021-09-02T09:24:48Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![a1114066506](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@a1114066506](https://discuss.elastic.co/u/a1114066506)\
**Post date:** [September 2, 2021, 9:24am UTC](https://discuss.elastic.co/t/does-logstash-can-automatically-consume-new-kafka-topic/283149/1 "2021-09-02T09:24:48Z")

</div>

I use input-kafka,configured by below code.

```auto
input{
  kafka{
    bootstrap_servers => "127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094"
    topics_pattern => "elk.*" 
    consumer_threads => 3
    decorate_events => true
    codec => "json"
    auto_offset_reset => "earliest"
    # latest
    group_id => "logstash1" 
}
}

```

But,when kafka clients create new topic,logstash can't consume the new topic. I must restart logstash or start with --config.reload.automatic and change the configuration file.

Does logstash can automatically consume new kafka topic?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 2, 2021, 1:16pm UTC](https://discuss.elastic.co/t/does-logstash-can-automatically-consume-new-kafka-topic/283149/2 "2021-09-02T13:16:09Z")

</div>

> [@a1114066506](#):
>
> Does logstash can automatically consume new kafka topic?

It should do so automatically. If you use topics\_pattern then the [consumer](https://kafka.apache.org/26/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#subscribe-java.util.regex.Pattern-) should re-evaluate it every 300 seconds by default.

---

<div class="post-metadata">

**Author:** ![a1114066506](https://avatars.discourse-cdn.com/v4/letter/a/d6d6ee/32.png) [@a1114066506](https://discuss.elastic.co/u/a1114066506)\
**Post date:** [September 3, 2021, 1:24am UTC](https://discuss.elastic.co/t/does-logstash-can-automatically-consume-new-kafka-topic/283149/3 "2021-09-03T01:24:02Z")

</div>

Thanks for your help 💯. max\_poll\_interval\_ms is useful .

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 1, 2021, 1:24am UTC](https://discuss.elastic.co/t/does-logstash-can-automatically-consume-new-kafka-topic/283149/4 "2021-10-01T01:24:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.