# Logstash Kakfa is Slow and Rebalancing Often

**URL:** <https://discuss.elastic.co/t/logstash-kakfa-is-slow-and-rebalancing-often/241924>\
**Category:** Logstash\
**Created:** [July 20, 2020, 6:43pm UTC](https://discuss.elastic.co/t/logstash-kakfa-is-slow-and-rebalancing-often/241924 "2020-07-20T18:43:17Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dhinesh](https://avatars.discourse-cdn.com/v4/letter/d/8dc957/32.png) [@dhinesh](https://discuss.elastic.co/u/dhinesh)\
**Post date:** [July 20, 2020, 6:43pm UTC](https://discuss.elastic.co/t/logstash-kakfa-is-slow-and-rebalancing-often/241924/1 "2020-07-20T18:43:17Z")

</div>

Hi.,

I have logstash reading from Kafka topic with 3 paritions. The logstash is slow in processing the data and kafka is rebalancing very often, which is causing duplication of records. I have tried to increase the max\_poll\_timeout =600000ms and session\_timeout = 300000ms with no luck. I need some guidance to solve the slowness problem.

```auto
    **Logstash config:**

    input {
      kafka {
        bootstrap_servers => '1.1.1.1:8989'
        group_id => 'Test'
        topics => 'Topic1'
        codec => json
        max_poll_records => '50'
       }
    }

    filter {
      ruby {
        code => "
          tstamp = event.get('epoch')
          event.set('Time',Time.at(tstamp.to_i).strftime('%Y-%m-%d %H:%M:%S'))
          tstamp.clear
        "
      }
    }

    output {

      stdout { codec => rubydebug { metadata => true } }

      csv {
        file_mode => 0644
       
        fields => ['Time','Field1','Field2','Field3']
        path => '/csv/data.csv'
      }
      
      }
    }

    **Logstah.yaml**

    http.host: "0.0.0.0"
    log.level: info
    path.logs: /logs
    pipeline.workers: 3
    pipeline.batch.size: 10000
    config.reload.automatic: "true"
    config.reload.interval: "3s"
    path.data: /data

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 17, 2020, 6:52pm UTC](https://discuss.elastic.co/t/logstash-kakfa-is-slow-and-rebalancing-often/241924/2 "2020-08-17T18:52:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.