Does logstash can automatically consume new kafka topic?

Elastic Stack Logstash
1

I use input-kafka,configured by below code.

input{
  kafka{
    bootstrap_servers => "127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094"
    topics_pattern => "elk.*" 
    consumer_threads => 3
    decorate_events => true
    codec => "json"
    auto_offset_reset => "earliest"
    #  latest
    group_id => "logstash1" 
}
}

But,when kafka clients create new topic,logstash can't consume the new topic. I must restart logstash or start with --config.reload.automatic and change the configuration file.

Does logstash can automatically consume new kafka topic?

2

It should do so automatically. If you use topics_pattern then the consumer should re-evaluate it every 300 seconds by default.

3

Thanks for your help :100:. max_poll_interval_ms is useful .

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.