# Logstash with kafka plugin not get messages in a new topic

**URL:** <https://discuss.elastic.co/t/logstash-with-kafka-plugin-not-get-messages-in-a-new-topic/238655>\
**Category:** Logstash\
**Created:** [June 25, 2020, 11:41am UTC](https://discuss.elastic.co/t/logstash-with-kafka-plugin-not-get-messages-in-a-new-topic/238655 "2020-06-25T11:41:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Altamir\_Dias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/altamir_dias/32/69586_2.png) [@Altamir\_Dias](https://discuss.elastic.co/u/Altamir_Dias)\
**Post date:** [June 25, 2020, 11:41am UTC](https://discuss.elastic.co/t/logstash-with-kafka-plugin-not-get-messages-in-a-new-topic/238655/1 "2020-06-25T11:41:58Z")

</div>

Hello guys,

I need a help in specific case. My environment have a DLL application who create messages and topics in Kafka, than logstash get messages of the all topics. But problem is new topic only collect when a next is create. I tested propertities auto\_offset\_reset but not work. Someone have idea how solve this problem?

[logstash.conf]

````auto
 input{
             kafka{
                     topics_pattern => ".*"
                     bootstrap_servers => "SERVER_IP_01:PORT"
                     sasl_jaas_config => "org.apache.kafka.common.security.plain.PlainLoginModule required username='${KK_USR}' password='${KK_PWD}';"
                     security_protocol => SASL_PLAINTEXT
                     sasl_mechanism => PLAIN
                     codec => "json"
             }
             kafka{
                     topics_pattern => ".*"
                     bootstrap_servers => "SERVER_IP_02:PORT"
                     sasl_jaas_config => "org.apache.kafka.common.security.plain.PlainLoginModule required username='${KK_USR}' password='${KK_PWD}';"
                     security_protocol => SASL_PLAINTEXT
                     sasl_mechanism => PLAIN
                     decorate_events => true
                     codec => "json"
                     auto_offset_reset => "earliest"
             }
     }
     output {
             elasticsearch{
                     hosts => ["ServerElasticsearch-01:9200", "ServerElasticsearch-02:9200", "ServerElasticsearch-03:9200"]
                     password => "${ES_PWD}"
                     #cacert => "/etc/logstash/logstash.pem"
                     index => "%{+YYYY.MM.dd}"
                     #ssl => true
                     #ssl_certificate_verification => true
                     user => "${ES_USR}"
             }
             stdout {
                     codec => rubydebug {metadata => true}
             }
 }
    ```
````

---

<div class="post-metadata">

**Author:** ![Altamir\_Dias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/altamir_dias/32/69586_2.png) [@Altamir\_Dias](https://discuss.elastic.co/u/Altamir_Dias)\
**Post date:** [July 7, 2020, 11:23pm UTC](https://discuss.elastic.co/t/logstash-with-kafka-plugin-not-get-messages-in-a-new-topic/238655/2 "2020-07-07T23:23:49Z")

</div>

Good evening,

After adition the properties in input: id and group\_id, my problem was solved.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 4, 2020, 11:23pm UTC](https://discuss.elastic.co/t/logstash-with-kafka-plugin-not-get-messages-in-a-new-topic/238655/3 "2020-08-04T23:23:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.